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Abstract 

The  objective  of  this  research  was  to  examine  the  job 
performance  of  the  U.S.  Air  Force  technical  order  (TO) 
managers  in  the  TO  acquisition  process  and  to  identify  ways 
to  improve  their  effectiveness  with  the  end  result  of 
improving  TOs. 

A  telephone  survey  of  TO  managers  performing  TO 
acquisition  within  Air  Force  Systems  Command  was 
accomplished.  Identification  of  personnel,  lack  of 
selection  criteria,  amount  of  formal  training,  interaction 
of  functional  offices,  and  the  level  of  top  management 
awareness  were  found  to  be  the  most  significant  issues 
facing  the  TO  manager.  TO  manager  composites  were  formed 
for  each  of  the  four  product  divisions  (Armament  Division, 
Aeronautical  Systems  Division,  Ballistics  Missile  office. 
Electronics  Systems  Division).  Finally,  a  composite  for  the 
entire  Air  Force  Systems  Command  was  formed. 

Recommendations  to  improve  the  effectiveness  of  the 
TO  manager,  based  on  the  formation  of  a  three  leveled 
Central  Technical  Order  Management  Center  (CTOC),  were 
presented.  Within  this  three  leveled  CTOC  framework,  the 
following  recommendations  were  made: 

1.  The  establishment  of  a  career  field  for  senior 
TO  acqusition  personnel  based  on  experience  and 
performance  within  the  field. 


The  identification  of  TO  managers  by  utilizing  a 
Special  Duty  Identifier  (SDI). 

The  establishment  of  selection  criteria  based  on 
formal  management  training  and  education;  prior 
TO  and  management  experience;  and  the  size, 
complexity,  and  acquisition  phase  of  the 
program. 

The  utilization  of  an  integrated  system  of  TO 
acquisition  management  courses  and  on-the-job 
training. 

The  continuation  of  recent  attention  and 
additional  support  by  top  management  to  the 
problems  of  TO  acquisition. 


UNITED  STATES  AIR  FORCE  TECHNICAL  ORDER  ACQUISITION: 
WHAT  CAN  BE  DONE  TO  IMPROVE  THE  EFFECTIVENESS 
OF  THE  TECHNICAL  ORDER  MANAGER? 


I.  Introduction 


General  Issue 


The  acquisition  process  has  traditionally 
concentrated  its  efforts  in  the  areas  of  cost,  schedule  and 
performance  when  acquiring  a  new  weapon  system.  Lt.  General 
Marquez,  Deputy  Chief  of  Staff,  Logistics  and  Engineering  in 
a  Project  Warrior  Distinguished  Speaker  presentation  at  the 
Air  Force  Institute  of  Technology  compared  the  acquisition 
process  to  a  stool  supported  by  three  legs  (21).  But  recent 
attention  in  the  area  of  supportabil ity  has  added  a  fourth 
leg  to  this  stool,  with  increased  emphasis  on  life  cycle 
costs.  The  largest  portion  of  life  cycle  costs  occur  after 
the  weapon  system  is  produced  and  in  service.  The  costs  are 
incurred  in  keeping  the  weapon  system  functioning  or  in  real 
terms,  "operational  readiness".  The  maintenance  effort 
cannot  occur  without  specific  direction  being  available  to 
the  maintainer;  this  direction  is  provided  in  the  form  of 
technical  orders  (TOs).  The  TO  is  one  of  the  keys  to  a 
successful  maintenance  effort.  Therefore,  management  must 
concentrate  the  needed  effort  to  ensure  that  TOs  procured  in 
the  acquisition  process  are  accurate,  timely,  cost  effective 


and  useable  ( 10  )  . 


The  TO  is  not  only  one  of  the  keys  to  a  successful 
maintenance  operation,  the  TO  is  also  one  of  the  primary 
tools  of  the  operator.  "Technical  Orders  are  the  key  to  the 
transfer  of  essential  technical  information  from  design 
engineers  to  operators  and  technicians"  (7:1).  The 
combination  of  operator  and  technician  performance  is  what 
determines  operational  readiness.  The  highest  operational 
readiness  possible  is  the  goal  of  the  Air  Force,  and 
technical  orders  impact  directly  the  successful  achievement 
of  that  goal.  General  Kelley,  Vice  Commander  of  the 
Tactical  Air  Command,  in  a  message  dated  21  January  1985, 
stated : 

"Operational  readiness  of  the  tactical  air  forces 
suffers  because  formal  technical  data  for  aircraft, 
weapons,  and  support  equipment  are  rarely  available 
when  operational  testing  is  performed  or  when  the 
weapons  are  fielded".  [20:1] 

Exploratory  research  has  found  that  the  acquisition 

process  in  many  cases  has  not  procured  technical  orders 

which  are  accurate,  timely,  useable  and  cost  effective. 

Many  possible  causes  for  the  problems  with  technical  order 

acquisition  have  been  explored.  This  thesis  will  examine  TO 

acquisition  from  a  management  perspective.  The  central 

figure  in  the  whole  TO  acquisition  process  is  the  TO 

manager.  The  TO  manager  is  tasked  by  TO  00-5-1  to: 

A.  Develop  as  early  in  the  acquisition  cycle  as 
possible  a  Technical  Publication  Development 
Management  Plan. 

B.  Evaluate  the  contractor  prepared  TO  Publication 


C.  Contract  for  a  TO  status  report  of  scheduled 
events . 

D.  Chair  and  conduct  in-process,  pre-publication  and 

post-publication  reviews.  [14:5-6] 

These  are  just  a  few  of  the  responsibilities  levied  against 

the  TO  manager.  It  is  clear  just  from  these  few 

responsibilities  that  a  TO  manager  ideally  should  be  a 

multi-talented  individual.  In  fact,  in  a  course  offered  by 

the  Air  Force  Institute  of  Technology,  School  of  Systems  and 

Logistics,  titled  "Technical  Order  Acquisition  and 

Management"  the  text  states  that  a  TO  manager  should  be  a 

proficient  technician,  an  experienced  planner,  an  effective 

technical  writer,'  a  proficient  negotiator,  a  coordinator,  a 

budget  analyst  and  a  cost  control  specialist  but  most 

importantly  an  effective  manager  (24). 


General  Problem 

Based  on  a  review  of  the  literature  on  TO 
acquisition,  a  further  examination  of  the  TO  manager  is 
warranted.  The  TO  manager  will  be  profiled  from  initial 
selection,  training,  qualifications,  experience,  interface 
with  other  agencies,  duties,  continuity  and  other  integral 
ingredients  with  an  end  purpose  of  determining:  What  can  be 
done  to  improve  the  effectiveness  of  the  TO  manager? 


Background 

The  design  of  a  research  project  attempting  to 
improve  the  effectiveness  of  the  TO  manager  must  consider 
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the  following  issues.  First,  examine  the  system  within 
which  the  TO  manager  operates.  Second,  note  the 
responsibilities  of  the  TO  manager  as  specified  in  the  Air 
Force  directives  and  policy  guides.  Third,  identify  the 
problems  related  to  the  TO  manager  and  their  recommended 
solutions . 

A  review  of  the  literature  shows  the  technical  order 
acquisition  process  is  an  integrated  subset  of  the  overall 
major  weapons  system  acquisition  process  (10).  In  each  of 
the  five  phases  of  the  acquisition  process;  Pre-Conceptual 
Phase,  Conceptual  Exploration  Phase,  Demonstration  and 
Validation  Phase,  Full-Scale  Development  Phase,  and 
Production/Development  Phase,  the  TO  manager  has 
responsibility  for  integrating  TO  requirements.  Failure  to 
effectively  integrate  TO  requirements  in  all  phases  will 
result  in  inadequate,  costly,  and  late  TOs  to  the  user  (10). 

The  responsibilities  of  the  TO  manager  are  not  listed 
in  a  single  publication,  but  rather  are  sprinkled  throughout 
volumes  of  regulations,  publications,  guidelines  and 
handbooks.  Three  publications:  Air  Force  Technical  Order 
System  (AFR  8-2),  Air  Force  Technical  Order  System  (TO  00-5- 


and  Technical  Publications  Acquisition  Manual  ( AFSCM 


310.2)  when  combined  provide  the  clearest  picture  of  the  TO 
manager's  many  responsibilities  (13,14,15). 

The  system  has  several  problems  which  contribute  to 


the  effectiveness  or  lack  of  effectiveness  of  the  individual 


TO  manager.  These  problems  will  be  examined  in  detail  based 
on  previous  research  by  Brown  and  Lyon.  They  were: 


1.  Need  for  better  communication  and  coordination  in 
the  TO  acquisition  process  between  the  various 
participants ; 


2.  Better  manning  for  TO  managers; 

3.  Better  training  for  TO  managers; 

The  Brown  and  Lyon  recommendations  to  solve  these  problems 


were : 


1.  Establishment  of  a  central  technical  order 
agency; 

2.  "Skeleton"  documents  of  statement  needs,  requests 
for  proposal  and  contracts; 

3.  A  handbook  identifying  coordination  and 
communication  responsibilities; 

4.  A  TO  acquisition  AFSC  or  career  field; 

[7:74] 


Definitions 


For  purposes  of  this  research  the  following  terms  are 
def ined : 

1.  Technical  order  will  refer  only  to  the  technical 
manual/publications  used  to  maintain  or  operate 
Air  Force  equipment. 

2.  Technical  order  manager  or  TO  manager  will  be 
used  to  identify  that  individual  tasked  with  the 
major  responsibility  of  technical  order 
acquisition.  It  is  not  a  title  per  se  but  a 
function . 

3.  Technical  Order  Management  Agency  (TOMA)  is  the 
organization  or  office  having  overall  management 
responsibility  for  the  acquisition  of  technical 
orders  required  for  the  operation  and  maintenance 
of  specific  equipment,  items,  or  modification 
thereto  (0:29).  Throughout  the  literature  the 
term  TOMA  is  interchangeably  used  to  describe 
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both  the  agency  and  the  individual  technical 
order  manager.  Therefore,  this  thesis  will  do 
the  same  when  directly  quoting  from  the 
literature . 

4.  Technical  order  management  effectiveness  is  the 
degree  to  which  the  TO  manager  operating  within 
the  existing  weapon  system  acquisition  process 
and  its  integrated  subset,  the  TO  acquisition 
process,  procures  a  TO  that  is  accurate,  timely, 
useable,  and  at  the  lowest  possible  cost  (23). 

5.  A  major  weapon  system  acquisition  is  defined  as 
$200  million  Research,  Development,  Test  and 
Evaluation  ( RDT&E )  or  $1  billion  procurement 
(12:6) . 

6.  Top  management  refers  to  any  position  within  the 
acquisition  process  higher  than  the  technical 
order  manager  (i.e.  PM,  HQ  AFSC,  HQ  USAF). 


Assumption 

TO  management  effectiveness  should  be  measured  by 
useability,  cost,  timeliness  and  accuracy.  Presently,  none 
of  the  TO  management  guidelines  and  regulations  specify 
measures  of  TO  management  effectiveness.  Yet,  as  the 
literature  review  establishes  there  are  factors  concerning 
the  TO  manager  that  have  an  impact  on  TO  management 
effectiveness.  These  factors  are;  identification  of 
personnel  as  TO  managers,  their  selection,  training, 
management  organization  and  top  management  support. 
Therefore,  this  thesis  makes  the  assumption  that  by 
improving  these  five  factors,  the  effectiveness  in  terms  of 
useability,  cost,  timeliness  and  accuracy  should  improve. 


Scope  of  Research 

This  research  project  will  concentrate  on  those  TO 
managers  involved  only  in  initial  acquisition  of  weapons 
systems.  The  TO  managers  are  primarily  Air  Force  Systems 
Command  (AFSC)  (or  Air  Force  Acquisition  Logistics  Center 
(AFALC)  personnel).  Air  Force  Logistics  Command  TO  manager 
personnel  while  involved  in  some  acquisition  efforts  are 
primarily  concerned  with  support  considerations  after 
Program  Management  Responsibility  Transfer  ( PMRT )  and 
therefore  will  not  be  dealt  with  in  this  research. 

This  research  project  readily  admits  there  are  other 
problems  with  the  acquisition  process  in  general  and  with  TO 
acquisition  in  particular.  It  is  not  the  intention  of  this 
effort  to  identify  additional  problems  or  even  to  identify 
interrelationships  between  the  many  problems.  This  thesis 
will  narrow  its  exploration  to  one  small  area  of  the 
problem,  examine  it  as  completely  as  possible  and  hopefully 
gain  enough  insights  to  make  realistic  recommendations  to 
remedy  the  problem.  The  five  problem  areas  that  will  be 
studied  by  this  research  project  are: 

1.  There  is  no  adequate  system  for  identifying  TO 
managers . 

2.  There  are  no  standards  or  criteria  for  selecting 
individuals  for  TO  manager  positions. 

3.  Training  for  TO  managers  is  inadequate. 

4.  Interaction  between  the  TO  manager  and  other 
functional  offices  is  inadequate. 
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5.  Top  management  awareness  of  TO  acquisition 
problems  is  not  adequate. 

The  research  is  a  further  exploration  of  the  work 
begun  by  Captain  Brown  and  Captain  Lyon.  The  intent  of 
their  research  was  "to  identify  the  nature  of  the  problems 
with  the  technical  order  acquisition  process  and  to  identify 
potential  changes  to  the  acquisition  system  that  can  solve 
these  problems"  (7:6).  One  of  their  goals  was  to  promote 
further  research  in  the  technical  order  acquisition  area. 
This  research  effort  is  a  result  of  the  Brown  and  Lyon  work. 

Research  Objective 

The  objective  of  this  research  is  to  examine  TO 
managers  in  the  technical  order  acquisition  process  and 
identify  ways  to  improve  their  effectiveness  with  the  result 
of  improving  technical  orders.  This  objective  will  be 
accomplished  by  examining  five  research  questions. 

Research  Questions 

The  five  research  questions  are: 

1.  How  are  TO  managers  initially  identified? 

2.  Can  the  establishment  of  selection  criteria  for 
TO  managers  improve  their  effectiveness? 

3.  Can  training  improve  the  effectiveness  of  TO 
managers  ? 

4.  Can  the  TO  manager's  interaction  with  other 
functional  offices  inside  and  outside  of  the 
System  Program  Office  be  changed  to  improve  the 
effectiveness  of  TO  managers? 

5.  How  do  TO  managers  perceive  the  level  of  top 
management  awareness  of  TO  acquisition  problems?" 


The  answers  to  these  five  questions  should  provide  a 
basis  for  recommendations  to  improve  the  TO  manager  s 
effectiveness  and  further  should  result  in  an  improvement  in 
the  technical  order  acquisition  process. 

The  next  chapter  will  consist  of  an  in-depth 
literature  review  about  technical  order  managers  and  the 
existing  technical  order  acquisition  process.  This 
literature  review  will  provide  the  basis  for  the  remainder 
of  this  research  project. 


II. 


Literature  Review 


This  literature  review  concerns  the  TO  acquisition 
process  and  the  TO  manager's  role  in  it.  It  identifies  the 
responsibilities  of  the  TO  manager  as  specified  in  Air  Force 
directives  and  policy  guides.  It  also  examines  the  research 
on  current  problems  related  to  the  TO  manager  and 
recommended  solutions  to  these  problems.  Finally,  the 
relationship  between  the  problems,  recommended  solutions  and 
the  objectives  of  this  research  will  be  discussed. 

Overview  of  TO  Acquisition  Process 

The  TO  acquisition  process  is  an  integrated  subset  of 
the  weapon  system  acquisition  process.  Since  TOs  are  an 
element  of  Integrated  Logistics  Support  (ILS),  it  is 
essential  that  the  planning  for  system  TOs  and  associated 
equipment  begin  during  the  early  planning  phases.  When 
timely  TO  planning  is  delayed  or  obstructed,  quality  is 
usually  sacrificed  and  procurement  costs  go  up;  therefore, 
ILS  costs  over  the  system's  life  cycle  go  up  (24:2).  The 
phases  of  the  weapon  system  acquisition  process  are;  Pre- 
Conceptual,  Conceptual  Exploration,  Demonstration  and 
Validation,  Full-Scale  Development,  and 
Production/Deployment . 

Pre-Conceptual  Phase 

The  Pre-Conceptual  Phase  begins  with  the  user 
preparing  a  Statement  of  Need  (SON).  TO  planning  should 
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begin  in  this  phase  because  the  initial  development  of  the 
operations  and  maintenance  concepts  for  the  weapon  system 
provide  the  basis  for  the  other  phases  of  the  acquisition 
process  (24:2).  The  maintenance  concept  identifies  the 
levels  of  maintenance,  skills  of  maintenance  personnel  at 
each  level,  the  amount  of  maintenance  that  will  be  performed 
and  the  environment  where  the  maintenance  will  be  performed 
(24:2).  The  activities  involved  in  the  maintenance  concept 
are  important  drivers  when  determining  the  types  and  numbers 
of  TOs  that  will  be  required  to  support  the  weapon  system 
and  related  support  equipment. 

The  purpose  of  involving  TOs  this  early  in  the 
acquisition  process  is  not  to  develop  specific  details,  but 
to  understand  the  factors  that  will  impact  TO  acquisition 
later  on  in  the  process  and  begin  planning  for  them  as  soon 
as  possible.  Figure  2.1  summarizes  the  key  factors  (24:2). 


Conceptual  Demonstration  Full-Scale  Production/ 
Exploration  &  Validation  Development  Deployment 
Phase  Phase  Phase  Phase 

Pre- 

Conceptual 

Phase 

Prepare  Statement  of  Need  (SON) 

*  State  mission 

*  Summarize  mission  need  in  operational  terms 

Key  tasks  to  be  performed 

Mission  purpose  and  capability 

Logistic  considerations 

Technical  publications  concept 
Maintenance  concept 

*  Summarize  deficiency 

-  Current  or  projected  inability  to  perform 
tasks 

Relationship  to  short  or  long  range 
capability  objectives 

Fig  2.1  Actions  Accomplished  During  Pre-Conceptual  Phase 

Conceptual  Exploration  Phase 

The  Conceptual  Exploration  phase  begins  with  the 
Secretary  of  Defense  Decision  Memorandum  and  the 
establishment  of  the  System  Program  Office  and  its  director, 
the  Program  Manager  (10).  The  purpose  of  this  phase  is  to 
identify  viable  alternatives  to  the  users'  needs.  Actions 
involving  TOs  are  exploratory  in  nature  but  are  very 
important.  According  to  Mr.  Munguia,  the  course  director 


for  the  Air  Force  Institute  of  Technology,  Technical  Order 
Acquisition  and  Management  Course, 


"The  key  to  developing  quality  publications  at 
competitive  cost  is  to  address  the  requirements 
during  concept  formulation  and  allow  the  process  to 
demonstrate  its  effectiveness  in  improving  overall 
system  supportability".  [24] 

Figure  2.2  shows  key  factors  that  should  be  considered  in 

this  phase  (24:3). 


Pre-  Demonstration  Full-Scale  Production/ 

Conceptual  &  Validation  Development  Deployment 

Phase  Phase  Phase  Phase 


Conceptual 

Exploration 

Phase 

applicability  of  new  technical  data  preparation  and 
display  techniques. 

applicability  of  current  technical  data  methods  and 
programs . 

applicability  of  commercial  data. 

improving  management  concepts  and  techniques  that  will 
allow  for  effective  tracking  of  data  cost. 

establishing  effective  coordination  procedures  during 
technical  publications  development. 

incorporation  of  a  good  quality  assurance  program  during 
development. 


Fig  2.2  Actions  Accomplished  During  Conceptual  Exploration 
Phase 


Demonstration  and  Validation  Phase 


Mr.  Munguia  describes  the  Demonstration  and 
Validation  Phase  as  "The  most  crucial  phase  in  the 
acquisition  process  in  terms  of  providing  the  interface 
necessary  between  system  design,  user,  and  the  logistics 
community"  (23:4).  This  phase  is  characterized  by  rapid 
expansion  of  the  system  project  office  as  managers  wrestle 
with  decisions  regarding  tradeoffs  between  performance 
requirements,  costs,  schedule  and  system  supportability.  As 
a  result,  functional  specialists  are  needed  to  manage  the 
identification  and  coordination  of  contractor  tasking 
requirements  with  the  user.  Air  Force  Logistics  Command, 
test  and  evaluation  organizations,  Air  Training  Command,  and 
others  as  the  system  dictates  (24:4). 

A  key  functional  specialist  is  the  TO  manager.  The 
TO  manager  is  chief  of  the  Technical  Order  Management 
Agency.  This  agency  has  overall  management  for  the 
acquisition  of  all  technical  publications  required  for  the 
installation,  operation  maintenance  and  overhaul  of  Air 
Force  weapon  systems,  support  equipment  and  materials 
(13:2) . 

Research  has  shown  that  TO  requirements  not 
identified  during  this  phase  of  the  systems  acquisition 
cycle  have  caused  substantial  increases  in  operational  and 
maintenance  costs  in  the  deployment  phase  of  the  process 
(  7  :  9  ;  1 1 :  2  4  ;  2  7  :  2  7  ) .  Therefore,  it  is  important  that  TOs  be 


developed  concurrently  with  the  total  program  effort  and 
that  TOs  are  compatible  with  the  overall  program  objectives 
(22:37).  Major  TO  requirements  that  are  the  responsibility 
of  the  TO  manager  are  listed  in  Figure  2.3  (24:4). 


Pre-  Conceptual  Full-Scale  Production/ 

Conceptual  Exploration  Development  Deployment 

Phase  Phase  Phase  Phase 


Demonstration 
&  Validation 
Phase 

Establish  points  of  contact  with  user,  logistic  support 
command,  training  command,  and  others  as  required. 

Prepare  and  coordinate  technical  publication  requirements 
for  draft  SOW. 

Coordinate  technical  publication  distribution 
requirements  for  early  training  and  test  functions. 

Review  and  resolve  technical  publication  conflicts  based 
on  input  from  data  call. 

Specify  technical  publication  requirements  in  final  SOW. 

Participate  in  contractor  selection  process. 

Host  technical  publications  guidance  conference  after 
selection  of  contractor. 


Fig  2.3  Actions  Accomplished  During  Demonstration  and 
Validation  Phase 


Ful 1 -Scale  Development  Phase 

During  the  Full-Scale  Development  Phase  program 
system  objectives  materialize  into  the  hardware  and 


prototypes  of  the  weapon  system  and  its  support  equipment. 


As  contractor  system/equipment  designs  become  firm,  designs 
and  procedures  for  operational  and  maintenance  tasks  are 
formalized  into  procedural  data  requirements  (24:6).  The 
contractor  in  this  phase  begins  the  preparation  of  draft 
manuals  to  be  used  in  contractor  validation  of  manuals. 
Also,  preliminary  manuals  for  ATC  training  requirements  are 
developed.  From  the  TO  manager's  point  of  view,  the  most 
important  consideration  underlying  all  TO  development  is 
that  quality  assurance  procedures  are  incorporated  into  the 
development  of  TOs  (28:28).  Factors  such  as  contractor 
validation,  government  in  process  review,  and  identification 
and  early  correction  of  problems  are  essential  for  an 
effective  TO  quality  assurance  program.  As  in  the  earlier 
phases  of  the  program,  the  TO  manager  must  work  closely  and 
coordinate  with  many  other  functional  offices  of  the  SPO, 
user,  test  and  evaluation  agencies,  AFLC,  ATC  and  others  as 
needed  to  develop  quality  TOs  (28:28).  Actions  which  the 
TO  manager  must  accomplish  or  insure  are  provided  are 
summarized  in  Figure  2.4  (24:7). 


Production/Deployment  Phase 


The  Production/Deployment  Phase  begins  when  the 
Secretary  of  the  Air  Force  is  satisfied  that  the  weapon 
system  developed  meets  the  Air  Force's  needs  and  he  gives 
the  go  ahead  (10).  In  this  phase,  TO  management  concerns 
are  still  focused  on  achieving  quality  assurance  for  the 
TOs.  As  the  contractor  begins  production  of  the  weapon 
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Pre-  Conceptual  Demonstration 
Conceptual  Exploration  &  Validation 
Phase  Phase  Phase 


Production/ 

Deployment 

Phase 


Full-Scale 

Development 

Phase 


Contractor  will  initiate  development  of  publication  in 
accordance  with  contract  requirements. 

Contractor  will  review  commercial  manuals  for 
applicability  or  possible  revision  to  comply  with 
government  requirements. 

Contractor  will  initiate  technical  publication 
validation. 

Contractor  will  formalize  Technical  Manual  Plan  ( TMP ) 
in  accordance  with  government  guidance. 

Technical  publications  manager  will  coordinate  specific 
quality  assurance  responsibilities  with  respective  DoD 
organizations. 

Technical  publications  manager  will  coordinate 
technical  publication  number  assignment. 

Technical  publications  manager  will  coordinate  in- 
process  review  conferences. 

Technical  publications  manager  will  arrange  for  early 
delivery  of  technical  publications  to  training  command 
and/or  test  and  evaluation  teams. 

Technical  publications  manager  will  arrange  for  effective 
accounting  and  distribution  of  publications  to  be  used 
for  verification. 

Technical  publications  manager  will  arrange  for  printing 
or  publication  through  the  government  printing  office. 


Fig  2.4  Actions  Accomplished  During  Full-Scale  Development 
Phase 
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system,  contractor  validation  is  still  continuing  and  Air 
Force  TO  verification  begins  on  those  TOs  validated  by  the 
contractor.  The  Air  Force's  test  and  evaluation  activities 
will  test  the  TOs  to  ensure  they  are  clear  and  adequate  for 
the  operation  and  adequate  for  the  operation  and  maintenance 
of  associated  equipment  and  for  certifying  that  the  TOs  are 
compatible  with  the  pertinent  hardware,  tools,  and  support 
equipment  (24:8).  All  deficiencies  noted  during 
verification  are  forwarded  to  the  contractor  to  make  the 
correct  changes  to  the  TOs.  Once  the  TOs  have  been  verified 
and  approved  the  TOs  are  formalized,  published  and 
distributed  to  the  user.  Specific  actions  taken  by  the  TO 
manager  during  this  phase  are  shown  in  Figure  2.5  (24:8). 

Remarks 

The  TO  acquisition  process  just  discussed  shows  the 
"ideal"  rather  than  the  actual  process.  The  difference 
between  the  two  consists  of  where  and  when  TO  manager 
actions  and  responsibilities  are  performed.  A  study  by 
Wigton  and  interviews  with  TO  managers  have  shown  that  many 
of  the  responsibilities  discussed  in  the  Demonstration  and 
Validation  Phase  are  actually  done  in  the  Full-Scale 
Development  Phase  (FSD)  ( 2 8 :2 8 ;2 7 ; 3 ; 9  ) .  In  particular  the 
TO  guidance  conference,  the  specification  of  TO  requirements 
in  the  formal  statement  of  work  and  the  contractor  selection 
process  are  usually  not  accomplished  until  FSD  (28:28). 
Also  many  of  the  planning  factors  for  TOs  discussed  in  the 
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Pre-  Conceptual  Demonstration  Full-Scale 

Conceptual  Exploration  &  Validation  Development 
Phase  Phase  Phase  Phase 

Production/ 

Deployment 

Phase 

Continuation  of  technical  publication  verification. 
Continuation  of  in-process  reviews. 

Pre-publication  reviews. 

Formal  inspection  and  acceptance  of  technical 
publications . 

Publication  and  general  distribution  of  formal 
technical  publications. 

Post  publication  reviews  of  technical  publications. 

Coordination  with  logistic  command  for  program 
management  responsibility  transfer  ( PMRT ) . 

Fig  2.5  Actions  Accomplished  During  Production/Deployment 
Phase 

Pre-Conceptua 1  and  Conceptual  Exploration  Phases  are  not 
done  until  the  Demonstration  and  Validation  Phase  (28:28). 

The  TO  acquisition  process  is  an  orderly,  sequential 
step  by  step  process  that  cannot  be  shortcut.  TO  planning 
begins  the  process  and  is  followed  by  TO  task 
identification,  TO  validation,  TO  verification,  TO  change 
requirements  and  lastly  printing.  If  these  steps  are  begun 
in  concept  exploration,  the  process  will  meet  the  ideal. 
But  if  the  planning  of  TO  acquisition  begins  farther  into 
the  acquisition  process,  everything  is  slipped  so  that 
something  downstream  will  suffer:  either  TO  accuracy, 
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timeliness,  cost  effectiveness  or  utility  (23).  A  study  by 
Brown  and  Lyon  found  that  beginning  planning  early  in  the 
process  was  not  a  source  of  TO  problems  in  itself  (7:55). 
They  did  find  a  lack  of  top  level  management  emphasis  on  TOs 
and  poor  communication  and  coordination  between  top 
management  and  SPOs  is  a  TO  problem  area  (7:56).  Their 
recommendation  to  establish  a  centralized  technical  order 


management  center  and  its  subordinate,  a  field  assistance 
division,  would  make  it  possible  for  technical  order 
acquisition  planning  to  begin  earlier  on  selected  programs. 
They  stated: 

If  field  assistance  personnel  would  be  assigned  to 
monitor  a  program  as  soon  as  the  program  is 
established,  then  technical  order  relevant  data  could 
be  compiled  from  the  beginning  of  the  program.  Even 
though  there  are  not  many  technical  order  acquisition 
related  activities  during  the  early  stages  of  a 
program,  the  field  assistance  division  would  be 
available  and  familiar  with  the  program  if  their 
assistance  is  needed.  [7:80] 

Therefore,  the  result  would  support  the  "ideal"  TO 

acquisition  process.  The  point  this  thesis  makes  is:  When 

TO  management  actions  deviate  from  the  "ideal”  TO 

acquisition  process,  the  more  difficult  it  is  to  get  quality 

TOs,  on  time,  at  the  lowest  possible  cost. 

The  overview  of  the  TO  acquisition  process 

highlighted  some  of  the  important  responsibilities  of  the 

TO  manager  and  the  way  these  responsibilities  are  integrated 

into  the  process.  The  next  section  will  review  Air  Force 


Currently,  there  is  no  single  source  that  can  be 
consulted  that  defines  in  detail  the  responsibilities  of  the 
TO  manager  (3).  Therefore,  this  review  will  examine  the  TO 
manager's  responsibilities  in  the  established  precedence  by 
effective  Air  Force  directives  and  policy  guides. 

Air  Force  Regulation  8-2,  Air  Force  Technical  Order 
System  is  the  publication  that  documents  official  policy 
concerning  the  Air  Force  TO  system  (13).  AFR  8-2  delineates 
the  responsibility  of  all  participants  in  the  TO  system  but 
does  not  specifically  identify  the  responsibility  of  the 
TO  manager. 

The  publication  next  in  precedence  is  TO-OO-5-1,  Air 
Force  Technical  Order  System  (14).  TO-OO-5-1  in  addition  to 
specifying  the  responsibilities  of  other  participants  in  the 
TO  system,  defines  the  responsibilities  of  the  TO  manager. 
The  following  is  a  summary  of  these  responsibilities 
compiled  by  Hatterick  and  Price  in  AFHRL-TR-80-50  (18:46): 

a.  Initiate  and  coordinate  the  pre-contract  planning 
for  procurement  of  TOs.  Ensure  that  all  affected 
commands  and  agencies  identify  TO-related 
requirements . 

b.  Prepare  Preliminary  TO  program  planning 
documentation,  to  include  the  work  statement  and  the 
preliminary  TO  Publications  Plan  (TOPP). 

c.  Conduct  and  chair  all  technical  publications 
conferences,  meetings,  reviews  (including  in-process, 


pre-publication,  and  post-publication),  and  other 
joint  efforts  related  to  the  program. 

d.  Ensure  that  adequate  arrangements  are  made  for  the 
participation  of  AFLC,  ATC,  using  commands,  and  other 
affected  agencies.  Arrange  for  Contract 
Administration  Office  (CAO)  representation  in 
meetings  with  the  contractor  and  procuring  activities 
as  required. 

e.  Provide  for  the  monitoring  of  the  contractor's 
validation  program.  Establish  and  manage  a 
verification  effort  as  either  a  one-step  or  two-step 
program,  and  coordinate  the  incorporation  of  needed 
changes  on  a  fast-reaction  basis. 

f.  Ensure  that  the  contractor  complies  with  all 
contractual  requirements,  applicable  specifications, 
standards,  exhibits,  maintenance  plans,  and 
provisioning  and  source  coding  actions  relative  to 
the  content,  format,  clarity,  level  of  writing, 
consistency,  adequacy,  and  accuracy  of  TOs . 

Next  in  precedence  is  Air  Force  Systems  Command 

Manual  310-2,  Technical  Publication  Acquisition  Manual  (15). 

This  publication  re-iterates  the  responsibilities  of 

TO-00-5-1  but  goes  into  more  detail  concerning  the  functions 

and  objectives  of  AFSC.  AFSCM  310-2  states: 

The  TOMA  will  initiate  planning  for  the  technical 
publications  development  concurrently  with  the  total 
program  planning  effort  to  ensure  that  the  management 
approach  is  compatible  with  the  overall  program 
objectives.  [15:2-1] 

AFSCM  goes  on  to  say: 

The  TOMA  is  responsible  for  the  accuracy  of  the 
technical  publications  . [15:2-2] 

The  format  of  AFSCM  310-2  breaks  out  the  specific 

TOMA  responsibilities  and  functions  during  the  TO 

acquisition  process  into  separate  sections  of  the 

publication.  These  sections  are  (15:2-1): 


a.  Requirements 

b.  Specifications 

c.  Budgeting  and  Funding 

d.  Work  Statements 

e.  Contracting 

f.  In-process  and  Pre-publication  Review 

g.  Scheduling  and  Reporting 

h.  Numbering,  Indexing,  Printing  and  Distribution 

i.  Validation  and  Verification 

j.  Inspection  and  Acceptance  Responsibilities 

k.  Engineering  Change  Proposal  (ECP)  and  Technical 
Publications  Revisions/Changes 

These  three  publications  provide  the  general  policy 
guidelines  and  responsibilities  of  the  TO  manager.  There 
are  several  other  directives  and  publications  which  also 
identify  these  same  general  responsibilities  in  addition  to 
more  detailed  instructions,  procedures  and  descriptions. 
But  all  are  described  and  discussed  in  the  work  of  Hatterick 
and  Price  and  need  not  be  reproduced  here  (18).  Their 


guidebook  fo:  TO  managers  titled  Technical  Order  Management 


and  Acquisition  is  the  most  comprehensive  analysis  and 
description  of  the  TO  acquisition  system.  The  objective  of 
their  research  as  described  in  the  introduction  states: 


This  guidebook  has  been  prepared,  in  two  volumes,  to 
be  a  basic  reference  publication.  It  is  intended  for 
use  by  all  individuals  involved  in  the  acquisition 
and  management  of  Air  Force  Technical  Orders  with 
emphasis  on  the  responsibilities  and  concerns  of  the 
TO  Management  Agencies  (TOMAs)  in  those  commands 
acquiring  TOs.  [18:1] 

Their  guidebook  is  the  result  of  a  two  year  study,  from  May 

1978  to  May  1  980,  on  the  TOMA  and  designed  to  be  a  handbook 

for  TO  managers  supplementing  AF  directives  and  publications 

(18:1). 


Research  on  TO  Manager  Related  Problems 


Research  on  TO  manager  related  problems  has  revealed 
several  problem  areas.  A  1984  study  by  Brown  and  Lyon 
identified  three  problem  areas  related  to  TO  managers.  They 
are  (1)  need  for  better  coordination  and  communication  in 
the  TO  acquisition  process  among  its  participants,  (2) 
better  manning  for  TO  managers,  and  (3)  better  training  for 
TO  managers  (7:53). 

Coordination  and  Communication.  The  Brown  and  Lyon 
recommendations  based  on  a  telephone  survey  of  TO  managers, 
Deputy  Program  Directors  for  Logistics  (DPMLs),  members  of 
the  Central  Technical  Order  Management  Group  (CTOM)  and  AFSC 
Program  Managers,  were  the  establishment  of  a  Central  TO 
Management  Center,  skeleton  documents  and  a  TO  manager 
handbook  (7:58). 

The  Central  TO  Management  Center  (CTOC)  would  be  a 
center  of  specialized  TO  acquisition  knowledge,  and  would 
assist  TO  managers  and  other  acquisition  agencies  with  TO 
information  and  problems  (7:27).  It  would  act  as  a  "central 
clearing  house"  for  the  TO  acquisition  process.  Currently 
the  Navy,  Army  and  Air  Force  Systems  Command  Armament 
Division  have  a  centralized  management  agency  (7:28). 
Specialists  in  these  agencies  have  been  assisting  their 
TO  managers  in  early  planning,  contracts,  and  establishing 
central  TO  acauisition  guidelines. 


A  1984  study  by  Wigton  on  the  need  for  a  centralized 

TO  center  also  concluded  that  the  Air  Force  needs  a  CTOC 

(28).  His  study  found  that: 

Air  Force  TO  responsibilities  are  not  vested  in  a 
single  organization  but  are  fragmented  among  and 
within  HQ  USAF,  AFLC,  AFSC  and  other  major  commands 
and  several  field  activities.  As  a  result  guidelines 
were  contradictory,  vague,  confusing,  non-directive 
and  biased  in  favor  of  major  (aircraft)  weapons 
systems  acquisition  programs.  [28:10] 

The  second  recommendation  of  "skeleton"  documents 

could  improve  coordination  by  communicating  possible  TO 

needs  to  individual  not  familiar  with  the  TO  process 

(28:29).  The  "skeleton"  documents  would  be  used  by 

individuals  working  in  the  Pre-Conceptua 1  and  Conceptual 

Phases  of  the  acquisition  process  where  there  is  normally  no 

TO  manager  assigned.  These  documents  would  assist  early 

planners  of  the  major  weapon  system  in  planning  for  TO 

requirements . 

The  third  recommendation  by  Brown  and  Lyon  about 
improving  coordination  and  communication  within  the  TO 
acquisition  process  is  the  establishment  of  a  TO  handbook 
(7:59).  The  handbook  would  be  dedicated,  or  have  sections 
dedicated  to  identifying  the  required  coordination  and 
communication  channels.  Then  TO  managers  would  have  a 
better  understanding  of  what  their  coordination  and 
communication  responsibilities  were. 

The  problem  of  coordination  and  communication  is  not 
new.  A  study  by  Carwise  and  Bemrose  in  1967  found: 


...the  management  information  system  for  the 
acquisition  of  technical  publications  to  be 
deficient.  There  is  little  standardization  of  status 
reporting  procedures,  status  information  received  is 
practically  useless  because  of  inaccuracies,  the 
system  does  not  provide  information  necessary  to 
avoid  procurement  of  duplicate  data,  and  there  is  not 
adequate  feedback  on  user  satisfaction.  [11:52] 

Their  recommendations  were: 

(1)  Technical  publication  managers  initiate  more 
aggressive  coordination  with  all  personnel  who  affect 
their  management,  (2)  An  improved  management 
information  system  be  established  to  aid  technical 
publication  managers,  (3)  a  study  be  initiated  to 
determine  the  feasibility  of  increasing  the  technical 
publication  manager's  control  over  the  elements  that 
effect  his  operation.  [11:53] 

Better  Manning  for  TO  Managers .  The  second  problem 

area  the  Brown  and  Lyon  study  found  was  that  manning  for  TO 

managers  was  inadequate.  Their  recommendations  for 

improving  the  manning  problems  were  (1)  establish  a  separate 

Air  Force  Specialty  Code  for  TO  managers,  and  (2)  establish 

a  CTOC  (7:61  ). 

The  Brown  and  Lyon  study  theorized  that  by 
establishing  a  separate  Air  Force  Specialty  Code,  manning 
problems  could  be  improved  (7:33).  Since  1  967  none  of  the 
recommendations  have  been  put  into  effect.  The 
establishment  of  a  CTOC  would  address  these  three  areas,  but 
currently  the  CTOC  is  not  a  reality.  In  a  preliminary  study, 
Brown  and  Lyon  found  that  TO  acquisition  personnel  at  the  AF 
Logistics  Management  Center  (AFLMC)  considered  manning  to  be 
a  number  one  problem  facing  the  TO  acquisition  process.  The 
AFLMC  people  stated:  "They  felt  that  there  is  no  standard 


or  criteria  for  assigning  TO  managers  to  a  program.  In 
fact,  many  programs  do  not  have  full  time  TO  managers" 
(7:23) . 

Research  by  Winn  and  Williams  also  found  this  to  be 
true.  They  stated: 

Overall  manning  of  the  ILS  (Integrated  Logistics 
Support)  effort,  including  the  Technical  Data 
Element,  is  significantly  below  that  requested. 

[29:10] 

Brown  and  Lyon  continued  to  state: 

Inadequate  manning  is  in  itself  a  problem,  and  is 
also  a  major  contributor  to  the  other  TO  acquisition 
problems.  Earlier  planning  in  the  TO  acquisition 
process  requires  adequate  manpower  to  accomplish  the 
job.  Better  coordination  and  communication  with 
other  agencies  also  requires  manpower.  [7:23] 

The  study  by  Wigton  also  found  TO  manning  to  be  a  problem 

for  all  AFSC  product  divisions  except  the  Armament  Division. 

The  study's  results  showed: 

Each  product  division  staff  assigned  one  person  to  TO 
acquisition  duties.  Personnel  assigned  were  not 
experienced  with  TO  acquisition...  TOMA  manning  with 
(certain)  SPOs  were  inconsistent,  and  not  based  on 
any  standard  factors  provided  by  manpower  officer, 
the  product  division  or  the  AFSC  headquarters  staff. 

[28:12] 

The  study  went  on  to  say  that  in  another  AFSC  product 
division : 

...no  TOMA  personnel  were  assigned  to  the  SPOs 
exclusively  for  TO  acquisition.  Instead,  TOMA  duties 
were  performed  by  personnel  responsible  for  support 
equipment  provisioning  as  well  as  TOs.  Several  of 
these  had  TO  acquisition  experience,  but  the 
background  of  most  was  in  other  logistics  support 
areas.  [28:12] 

The  Armament  Division  (AD)  of  AFSC  was  one  exception 

the  study  found.  It  had  TO  managers  exclusively  assigned  to 


SPOs  for  TO  acquisition.  This  was  because  AD  had  formed  a 

"divisional'  CTOC  to  manage  all  division  TO  related 

problems  including  manning  (3). 

The  recommendations  by  the  Wigton  study  are  similar 

to  the  Brown  and  Lyon  study.  The  Wigton  study  found: 

A  contributing  factor  to  inadequate  staffing  and 
manning  problems  was  the  lack  of  one  military  or 
civilian  career  field  for  TOMAs.  This  perpetrated 
inexperience  problems  within  SPOs  with  military  TOMA 
positions,  and  created  career  progression  problems 
for  civilian  TOMAs.  [28:12] 

Additionally,  civilian  TO  managers  believed  their  jobs  were 

'dead-end'  jobs.  For  several  years  the  CTOM  Group  has  been 

trying  to  establish  military  and  civilian  career  fields, 

without  success  (28:12). 

These  last  findings  are  supported  by  another  study, 
done  earlier.  Carwise  and  Bemrose  found  that  TO  manager  low 
retention  rates  were  because  the  job  itself  was  far  from 
enticing,  even  though  challenging  (11:45).  Their  study  also 
recommended  a  separate  AFSC  for  TO  managers  because: 

...an  established  career  field  would  increase  the 
attractiveness  of  assignment  in  the  data  management 
area.  Those  assigned  to  technical  publication 
management  positions,  for  example,  would  know  what 
type  and  level  of  assignment  follows.  At  present 
(1967)  technical  publication  managers  cannot 
anticipate  future  assignments  because  no  progression 
or  assignment  pattern  exists...  [11:45] 

A  feasibility  study  by  Peters  on  the  establishment  of 

an  AFSC  for  all  TO  managers  in  the  entire  TO  system  (System 

of  AFLC,  user  TO  management)  found  the  Air  Force  personnel 

system  would  not  support  a  separate  AFSC  for  the  following 


reasons:  limited  size  of  TO  population,  limited  career 

ladder  and  progression  opportunities,  and  an  extra 
bureaucratic  layer  (l:Al3-7).  Peters'  study  concluded  that 
the  following  alternatives  need  further  study: 

a.  establishment  of  a  separate  TO  management  AFSC 

b.  establishment  of  a  special  duty  identifier  for 
selected  TO  management  duties 

c.  establishment  of  an  AFSC  for  TOMAs  (AFSC  and 
AFLC ) 

d.  additional  overall  TO  training  for  all  TO  users, 

with  an  AFSC  shredout  to  identify  personnel 
performing  special  TO  duties.  [l:A13-7] 

The  second  alternative  recommended  by  the  Brown  and 

Lyon  study  to  solve  the  manning  problem  was  the 

establishment  of  a  CTOC.  Their  results  showed  that  "the 

establishment  of  a  CTOC  would  allow  smaller  programs  to 

effectively  purchase  TOs  without  the  assignment  of  a  TOMA" 

(7:28).  It  further  said  that  the  CTOC  could  assist  in  TO 

acquisition  needs  of  smaller  programs,  thus  reducing  the 

need  for  TOMA  manpower  in  those  programs.  The  end  result 

will  be  a  reduction  of  manpower  requirements  overall  (7:33). 

The  Wigton  study  also  concluded  that  a  CTOC  would  be 

a  great  asset  in  reducing  manning  problems.  If  established 

one  of  the  CTOC's  responsibilities  would  be  to  identify, 

track,  and  man  SPOs  and  AFLC  program  offices  with  qualified 

TO  managers  (28:23). 

Remarks.  Although  the  majority  of  the  research 
concluded  that  a  separate  Air  Force  Specialty  Code  for  TO 
managers  is  needed,  the  Peter's  study  disagreed  (l:A13-7). 
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None  of  these  studies  however,  described  how  to  establish  an 
Air  Force  Specialty  Code  or  explore  the  viable  alternatives 
recommended  in  the  Peter's  study.  The  subject  of  a  separate 
Air  Force  Specialty  Code  therefore  is  not  a  closed  issue. 
Any  effort  directed  at  improving  the  TO  acquisition  process 
must  take  into  account  the  manning  problems  discussed  and 
thus  further  explore  the  impact  of  an  Air  Force  Specialty 
Code  on  the  effectiveness  of  the  TO  manager. 

Additionally,  interviews  with  Munguia,  Breslin  and 
Weaver  concluded  that  new  personnel  being  assigned  TO 
manager  duties  should  have,  as  a  minimum,  several  years 
experience  as  TO  users,  especially  as  maintenance 
technicians  (22;26;  3).  Other  TO  managers  agreed  with  the 
experts'  conclusions.  Mr.  Cambell  and  Mr.  Burkhardt,  TO 
managers  at  ESD,  both  said  "a  TO  manager  must  have  equipment 
experience  first"  ( 8 ; 9 ) .  CMSgt  Breslin,  at  AD,  summed  up 
the  'experts'  recommendation  for  equipment  experience  by 
saying  "the  TO  manager  needs  to  translate  the  user  wants 
into  acquisition  requirements."  To  do  this  the  TO  manager 
has  to  know  both  worlds.  The  TO  manager  learns  the  user 
world  by  'hands-on'  equipment  experience.  The  acquisition 
world  can  be  learned  through  training  and  experience  in  the 
acquisition  community  (3). 

Better  Training  For  TO  Managers.  The  third  problem 
area  found  by  the  Brown  and  Lyon  study  was  that  TO  manager 
training  needed  improvement.  They  concluded  that: 


Training  appeared  to  be  the  largest  single  problem 
with  the  TO  acquisition  process.  In  fact  it  was  very 
difficult  to  separate  training  problems  from  manning 
problems  or  any  of  the  other  problems  investigated. 
Training  referred  to  much  more  than  formal  education. 
It  also  included  experience  and  "corporate 
knowledge."  [7:25] 

To  solve  this  problem,  the  Brown  and  Lyon  study 

recommended  (1)  establishment  of  a  separate  Air  Force 

Specialty  Code  or  career  field  for  TO  managers,  (2) 

establishment  of  a  CTOC,  (3)  establishment  of  a  handbook  on 

coordination  and  communication  responsibilities  (7:65). 

In  the  study's  recommendation  of  a  CTOC,  the  CTOC 

would  be  subdivided  into  three  divisions.  One  of  those 

divisions,  the  Field  Assistance  Division,  would  have  the 

responsibility  for  establishing  a  training  program  for  TO 

managers  and  monitoring  training  programs  of  Air  Force 

Systems  Command  product  divisions  and  SPOs.  The  Air  Force 

Specialty  Code  and  handbook  recommendation  previously 

discussed  would  be  integrated  into  the  training  process 

(7:79).  The  study  did  not  recommend  a  type  of  manual  or 

requirements  for  the  training  program. 

The  study  by  Wigton  also  recommended  that  the 

CTOC  have  the  responsibility  for  a  training  program  (28). 

Again,  their  study  did  not  identify  any  particular  program 

or  curriculum  other  than  training  in  general  as  a  problem 

area. 

Remarks.  Currently  there  are  two  formal  training 
programs  for  TO  managers.  The  first  is  the  Air  Force 
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Institute  of  Technology  (AFIT)  Technical  Order  Acquisition 
and  Management  Course  (SYS  230)  taught  by  Mr.  Munguia.  The 
second  is  a  training  program  set  up  at  AD  for  AD  TO  managers 
by  CMSgt  Breslin  ( 2  3 ;  3 ) .  The  AFIT  course,  open  for  TO 
managers  and  other  personnel  with  jobs  involving  TO 
management  is  two  weeks  long.  As  reported  by  Brown  and 
Lyon,  Mr.  Munguia  stated: 

The  knowledge  and  expertise  required  to  effectively 
manage  a  technical  order  acquisition  program  is 
immense.  In  a  two  week  course,  I  cannot  do  much  more 
than  give  an  overview  of  the  major  topic.  The  best 
the  students  can  do  is  learn  of  their 
responsibilities  and  hopefully  where  to  go  to  when 
they  need  assistance  with  the  details.  [6] 

Important  to  this  problem  area  is  that  "corporate 

memory"  is  needed  to  define,  plan,  and  execute  a 

comprehensive  training  program.  A  general  overview  course, 

similar  to  the  AFIT  SYS  230  course  is  needed  but  a  more  in 

depth  and  specific  course  such  as  the  program  at  AD  is  also 

needed  at  all  AFSC  product  divisions.  Furthermore  the 

training  program  should  include  the  Technical  Order 

Management  Agency  itself  to  train  new  people  on  the 

peculiarities  of  that  agency's  functions  and  special  TO 

requirements  (23; 27; 28). 

Conclusions 

The  literature  has  shown  the  process  of  the  TO 
acquisition  is  a  subset  or  subsystem  of  the  larger 
acquisition  system.  The  previous  research  has  identified 


the  problems  with  the  TO  acquisition  process  and  provided 


recommended  solutions.  The  Brown  and  Lyon  study  concluded 
that  the  problems  are  interrelated.  Treating  one  problem, 
say  training,  will  have  profound  effects  on  other  elements, 
such  as  manning  or  system  communication  and  coordination 
(7:75).  The  establishment  of  a  CTOC  as  both  the  Brown  and 
Lyon  study  and  the  Wigton  study  advocated  was  a  central  key 
to  solving  the  problems  of  the  TO  acquisition  process  (7:75; 
28).  The  authors  of  this  research  project  agree.  But  as 
the  literature  has  also  shown  little  has  been  done  to  learn 
and  understand  the  elements  involved  with  the  human  aspect 
of  the  problem,  the  technical  order  manager.  This  research 
project,  as  described  in  the  scope,  will  focus  on  the  TO 


manager , 


The  literatature  review  leads  to  the  conclusion  that 


the  identification  of  TO  managers  is  a  problem.  This  can  be 
seen  by  all  prior  research  advocating  that  an  Air  Force 
Specialty  Code  is  needed  for  TO  managers.  Peters  points  out 
an  Air  Force  Specialty  Code  is  only  one  alternative  to  the 
identification  problem.  Other  alternatives  he  points  out 
are  an  SDI,  and  an  Air  Force  Specialty  Code  for  joint 
acquisition  TO  managers  (l:Al3-6).  A  final  viable 
alternative  is  that  no  Air  Force  Specialty  Code  is  needed  at 
all,  but  the  control  of  TO  manager  manning  can  be  done  "in 
house"  through  the  CTOC  as  Wigton  and  Brown  and  Lyon  point 
out  (7;28). 


To  define  the  problem  further,  five  independent 
variables  emerge  (16).  First  is  the  identification  process 
itself  and  second,  the  criteria  used  to  select  individuals 


for  TO  manager  positions.  This  research  project  will 
explore  both  these  variables  by  asking  two  research 
questions : 

1.  How  are  TO  managers  initially  identified? 

2.  Can  the  establishment  of  selection  criteria 
for  TO  managerss  improve  their  effectiveness? 

The  third  variable  the  literature  review  discussed 

was  the  problem  of  training.  It  identified  only  two  formal 

training  programs  that  are  currently  in  being.  Both 

programs  are  contributing  to  the  improvement  of  TO  manager 

effectiveness  but  on  a  limited  basis.  The  AFIT  course  as 

Mr.  Munguia  said  "only  makes  TO  managers  aware  of  their 

responsibilities"  (6).  The  AD  program  that  CMSgt  Breslin 

established,  is  only  for  AD  TO  managers  (5:6). 

Air  Force  Manual  50-62,  Principles  and  Techniques  of 

Instruction  states: 

Training  is  the  factor  of  management  that  insures 
standard  procedures,  overall  efficiency,  and  mutual 
understanding.  It  is  the  medium  used  by  the 
executive  or  commander  to  pass  on  the  why,  what,  and 
how  of  the  job  to  be  done.  [17:1] 

The  literature  shows,  initially,  that  the  majority  of 

acquisition  TO  managers  are  not  understanding  the  why,  what 

and  how  of  the  TO  manager  job.  Therefore  this  thesis  will 

explore  further  the  training  aspect  of  the  TO  manager.  It 

will  do  it  by  answering  the  research  question: 


3.  Can  training  improve  the  effectiveness  of  the 
TO  manager? 

A  fourth  independent  variable  this  research  project 
will  examine  is  the  interaction  of  other  functional  offices 
with  the  TO  manager.  The  Brown  and  Lyon  study  as  well  as 
the  Wigton  study  found  coordination  and  communication  of  the 
TO  manager  with  elements  outside  its  functional  area  to  be 
poor  (7:58;28:18). 

Based  on  the  description  of  the  TO  managers' 
participation  in  the  major  weapons  system  acquisition 
process  and  a  review  of  directives  and  policy  guides  on  the 
TO  manager  responsibilities,  it  can  be  concluded  that  much 
of  what  the  TO  manager  does  requires  extensive  interaction 
with  other  functional  areas.  Therefore,  this  thesis  will 
answer  the  research  question: 

4.  Can  the  TO  managers'  interaction  with  other 
functional  offices  inside  and  outside  of  the  SPO 
be  changed  to  improve  the  effectiveness  of  the 
TO  manager? 

Finally,  a  fifth  independent  variable  identified  in 
the  literature  review  which  needs  further  study  is  in  the 
area  of  top  management's  lack  of  emphasis  on  TO  problems. 
Carwise  and  Bemrose  reported  in  1967  that  top  management  did 
not  give  enough  attention  to  the  TO  acquisition  process 
(11:53).  Wigton,  Brown  and  Lyon  studies  also  point  this  out 
(  2  8  : 1  3  ;  7  :  2  2  ) .  Furthermore  it  can  be  seen  from  the  fact  TO 
managers  are  not  assigned  to  the  SPO  in  some  cases  until  the 
demonstration  and  validation  phase  and  that  TO  acquisition 
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is  not  given  a  high  enough  priority.  Yet,  as  the  Brown  and 
Lyon  study  pointed  out,  the  poor  quality  of  TOs  the  user  has 
in  the  deployment  phase,  are  a  function  of  TO  acquisition 
problems  (7).  Thus,  to  affect  the  much  needed  changes  in 
the  TO  acquisition  process  including  the  establishment  of  a 
CTOC,  improved  training  programs,  better  defined  policy 
guidelines  and  an  established  identification  and  selection 
process  for  TO  managers,  top  management  must  support  these 
changes.  Therefore  this  thesis  will  answer  the  research 
question: 

5.  How  do  TO  managers  perceive  the  level  of  top 
management  awareness  of  TO  acquisition  problems? 

Chapter  Summary 

The  TO  acquisition  process  has  been  reviewed  and 
important  TO  manager  actions  have  been  identified  in  the 
acquisition  process.  TO  manager  responsibilities  defined  in 
Air  Force  directives  and  guides  have  been  identified  and  a 
review  of  current  research  on  the  TO  acquisition  process 
have  identified  five  TO  manager  related  problems.  These 
problems  as  identified  by  the  literature  review  are: 

1.  There  is  no  adequate  system  for  identifying 
TO  managers. 

2.  There  are  no  standards  or  criteria  for 
selecting  individuals  for  TO  manager  positions. 

3.  Training  for  TO  managers  is  inadequate. 

4.  Interaction  of  the  TO  manager  and  other 
functional  offices  is  inadequate. 


5.  Top  management  awareness  of  TO  acquisition 
problems  is  not  adequate. 

The  objective  of  the  research  is  to  answer  the  five 
research  questions  that  relate  to  the  five  problems.  The 
research  questions  are: 

1.  How  are  TO  managers  initially  identified? 

2.  Can  the  establishment  of  selection  criteria 
for  TO  managers  improve  their  effectiveness? 

3.  Can  training  improve  the  effectiveness  of 
the  TO  manager? 

4.  Can  the  TO  managers'  interaction  with  the  other 
functional  offices  inside  and  outside  of 

the  System  Program  Office  be  changed  to 
improve  the  effectiveness  of  the  TO  manager? 

5.  How  do  TO  managers  perceive  the  level  of  top 
management  awareness  of  TO  acquisition 
problems? 

The  next  chapter  will  define  the  methodology  for 
quantifying  the  five  research  questions. 


III.  Methodolo' 


Justification  of  Research  Method 


It  was  mentioned  in  the  Scope  section  that  this 
research  was  a  further  exploration  of  the  work  begun  by 
Captain  Brown  and  Captain  Lyon.  Due  to  the  exploratory 
nature  of  this  research  no  hypotheses  was  tested.  The 
literature  review  identified  five  problem  areas  which  this 
study  explored  in  further  detail.  The  purpose  of  this 
research  phase  was  to  examine  the  TO  manager's  role  in  the 
TO  acquisition  process  and  to  identify  ways  to  improve  his 
effectiveness . 

This  examination  was  accomplished  by  the  use  of 
telephone  interviews  with  functioning  TO  managers.  The 
interviewees  were  asked  to  respond  either  positively  or 
negatively  to  measurement  questions  derived  from  the  five 
research  questions.  The  interview  schedule  is  presented  as 
Appendix  A.  The  percentage  of  the  interviewees'  positive  or 
negative  responses  were  used  to  establish  the  validity  of 
the  research  questions  as  a  measure  of  TO  managers' 
effectiveness.  Additionally,  expanded  response  questions 
were  utilized  to  determine  information  not  reduceable  to 
positive  or  negative  responses  and  to  solicit  suggestions  or 
recommendations  to  improve  the  management  of  TOs. 

The  interview  used  structured  questions;  however,  one 
of  the  advantages  of  this  technique  was  that  responses  were 
expanded  as  necessary. 


Three  alternative  data  collection  methods  were 
considered  for  this  research  project:  (1)  on  site 

interviews,  (2)  mail  survey,  and  (3)  telephone  survey. 
While  on  site  surveys  were  the  most  flexible,  they  were  also 
the  most  expensive.  The  interview  population  for  this 
research  effort  was  spread  over  four  Air  Force  bases  located 
in  four  states,  making  the  cost  in  terms  of  time  and  money 
too  prohibitive.  Since  the  intent  of  this  research  was  to 
contact  as  nearly  as  possible  the  entire  TO  manager 
population,  the  non  response  rate  for  mail  surveys  was 
unacceptable  (16:308).  Telephone  interviews  have  reasonably 
high  response  rates,  using  the  autovon  system  eliminates  all 
dollar  costs,  and  interview  bias  was  reduced  "because  of  the 
lack  of  face-to-face  contact  between  interviewer  and 
respondent",  thus  making  telephone  interviews  the  most 
practical  measurement  tool  (16:306). 

Population  Description 

The  Brown/Lyon  thesis  upon  which  this  thesis  was 
predicated  divided  the  respondents  by  product  divisions. 
The  four  product  divisions  within  the  Air  Force  Systems 
Command  (AFSC)  which  dealt  with  the  acquisition  of  technical 
orders  were  the  Aeronautical  Systems  Division  (ASD),  the 
Electronics  Systems  Division  (ESD),  the  Armament  Division 
(AD),  and  the  Ballistic  Missile  Office  (BMO).  These  four 
divisions,  because  they  dealt  with  the  acquisition  of  AF 
weapon  systems  and  related  equipment,  were  within  the  scope 
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of  this  research.  "The  fifth  product  division  in  AFSC, 
Space  Division,  was  not  included  in  the  research  because  of 
the  unique,  non-weapon  system  nature  of  the  equipment 
acquired  by  that  division"  (7:38).  Furthermore,  because 
this  effort  was  examining  the  effectiveness  of  the  TO 
manager  within  the  four  product  divisions,  this  thesis  used 
the  description  of  the  organizational  structure  of  the  four 
divisions  as  cited  by  Brown  and  Lyon. 

A  description  of  the  organizational  structures 
of  the  four  divisions  is  necessary  to  understand  the 
nature  of  the  population.  These  structures  differ 
because  of  the  varying  scope  of  the  programs  for 
which  the  divisions  are  responsible.  For  example, 
ASD  deals  with  major  systems  acquisitions  such  as 
aircraft,  simulators,  engines,  and  related 
aeronautical  equipment,  while  ESD  and  AD  deal  with 
smaller  weapon  systems  or  subsystems  of  larger 
systems.  What  follows  is  a  general  description  of 
the  different  divisions'  System  Program  Office  (SPO) 
organizations,  recognizing  that  each  SPO  will  differ 
in  detail . 

In  ASD,  individual  SPOs  are  often  large 
organizations,  with  separate  offices  dedicated  to 
specific  acquisition  responsibilities,  and  these 
separate  offices  contain  specialists  in  the 
corresponding  areas.  In  general,  each  ASD  SPO  has 
three  levels  of  responsibility  regarding  TO 
acquisition.  At  the  upper  level  are  Program  Managers 
(PMs),  who  are  responsible  for  the  acquisition  of  the 
entire  system,  including  TOs.  Reporting  to  the  PM  is 
a  Deputy  Program  Manager  for  Logistics  (DPML) ,  who  is 
responsible  for  the  portion  of  system  acquisition 
that  relates  to  logistics,  again  including  TOs. 
Finally,  there  are  Technical  Order  Managers  (TOMAs), 
reporting  to  the  DPML,  that  coordinate  and  manage  the 
day-to-day  activities  of  TO  acquisition. 

In  contrast  to  the  large-scale  SPOs  typical 
of  ASD  are  the  SPOs  of  ESD  and  AD.  The  systems  these 
divisions  are  responsible  for  acquiring  are  typically 
smaller  in  scope  than  those  of  ASD,  with  a 
corresponding  reduction  of  dedicated  functions  within 
those  SPOs.  Neither  division  has  TOMAs  per  se;  in 
ESD  the  DPMLs  perform  TO  acquisition  as  part  of  their 
normal  duties,  while  in  AD  TO  acquisition  specialists 


are  matrixed  from  a  central  office  into  SPOs  as 
required.  The  organization  of  BMO  is  a  mix  of  the 
other  three;  the  SPOs  are  large  (as  in  ASD)  but  few 
in  number,  while  TO  acquisition  personnel  are 
matrixed  from  a  central  office  similar  to  the  ESD  and 
AD  structure.  [7:38-40] 

In  each  product  division,  respondents  were 

identified  as  those  individuals  tasked  with  the 

responsibilities  of  TO  acquisition  rather  than  by  a  formal 

duty  title.  The  method  used  to  generate  possible 

respondents  began  by  first  identifying  all  the  current 

systems  active  in  the  four  product  divisions.  A  data  base 

maintained  by  the  Air  Force  Acquisition  Logistics  center  at 

Wright-Patterson  AFB  was  used  for  this  purpose.  The  list  of 

current  systems  was  then  cross-matched  against  a  point  of 

contact  list  provided  by  each  product  division.  These  lists 

varied  in  completeness;  for  example  CMSgt  Breslin  at  AD  was 

able  to  provide  a  list  of  names  and  telephone  numbers  of 

every  TO  manager,  which  facilitated  the  interview  process 

greatly.  The  BMO  list  began  as  a  single  point  of  contact 

taken  from  a  list  of  attendees  at  the  Air  Force  Institute  of 

Technology,  System  230  course,  and  then  utilized  the 

telephone  survey  instrument  to  generate  the  remaining  names. 

ASD  required  a  bit  of  detective  work  to  identify  the 

individual  who  had  a  list  of  key  names  within  each  weapon 

system  acquisition  program.  By  contacting  the  individuals 

on  the  list  and  again  utilizing  the  telephone  interview,  a 

comprehensive  list  of  ASD  TO  managers  was  generated.  ESD 

presented  the  biggest  challenge.  Mr.  Herbert  Cambell 
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provided  a  list  of  DPMLs  and  PMs  grouped  by  program.  It  was 
necessary  to  contact  these  individuals,  explain  the  nature 
of  the  research,  and  obtain  their  approval  to  interview 
personnel  who  fit  this  research's  definition  of  a  TO 
manager.  The  ESD  personnel  tasked  with  TO  acquisition  are 
unofficially  termed  "loggies"  and  as  such  were  responsible 
for  all  logistics  tasks  in  addition  to  acquisition  of  TOs. 
These  "loggies"  worked  in  such  isolation  from  each  other 
that  it  was  only  rarely  that  the  interviewee  was  able  to 
provide  names  of  other  TO  managers. 

The  TO  manager  population  consists  of  109  personnel. 
Due  to  the  relatively  small  size  of  the  population,  the 
intent  of  the  research  was  to  conduct  a  census  rather  than  a 
sample.  The  following  is  a  list  of  identified  TO  managers 
in  each  of  the  product  divisions  and  the  number  of  them 
interviewed : 

Division  Identified  Interviewed 

AD  15  15 

ASD  43  41 

BMO  6  5 

ESD  45  40 

TOTAL  109  101 

The  differences  between  the  identified  numbers  of  TO 

managers  and  those  interviewed  were  due  to  personnel 

transfers  without  replacement,  SPO  recognization,  and 

individuals  who  were  unable  to  be  contacted  due  to  leave  or 

temporary  duty  during  the  time  the  interviews  were 
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conducted.  The  period  the  interviews  were  conducted  was 
from  March  1985  to  June  1985. 

The  Telephone  Interview 

The  telephone  interview  was  the  method  of 
interrogation  used  (see  Appendix  A).  The  instrument  began 
by  introducing  the  interviewer  to  the  prospective  respondent 
and  continued  with  a  brief  explanation  of  the  purpose  of  the 
interview  and  why  the  respondent  was  chosen  to  participate. 

The  questions  required  several  types  of  responses. 

1.  Numerical  (i.e.  dates,  number  of  personnel) 

2.  Historical  (i.e.  acquisition  phase  at  time  of 
assignment ) 

3.  Biographical  (i.e.  job  title) 

4.  Positive/Negative  (i.e.  yes-no  responses) 

5.  Expanded  Response  (i.e.  suggestions  or 
recommendations ) 

The  only  expected  separation  of  respondents  was  by 
product  division;  therefore,  no  particular  sequence  was  used 
in  conducting  interviews.  The  only  ordering  was  as  a  result 
of  the  names  provided  by  the  instrument.  The  anonymity  of 
respondents  was  preserved  by  annotating  the  response  sheet 
with  .the  title  of  the  product  division  only. 

According  to  Emory,  the  three  major  considerations  in 
evaluating  a  measurement  tool  are  practicality,  reliability 
and  validity  (16:128).  Practicality  was  previously 
discussed  in  the  section  explaining  the  rational  behind 
selecting  the  telephone  interview  as  the  survey  instrument. 


The  reliability  of  the  telephone  interview  schedule 
was  demonstrated  by: 

The  telephone  interview  schedule  was  given  a  trial 
run  to  demonstrate  whether  the  questions  contained  in  the 
instrument  would  generate  the  type  responses  necessary  for 
the  purposes  of  this  thesis.  The  trial  population  consisted 
of  twenty  Air  Force  Institute  of  Technology  School  of 
Logistic  Professional  Continuing  Education  students  whose 
normal  duties  involved  technical  order  management  in  one 
capacity  or  another.  The  students  were  familiar  with  the 
terms  and  language  normally  associated  with  technical  orders 
and  were  enrolled  in  Technical  Order  Acquisition  Management 
SYS  230.  The  trial  population  responses  from  all  personnel 
not  functioning  as  technical  order  managers,  as  defined  in 
this  thesis,  were  unuseable.  The  six  respondents  actually 
performing  TO  manager  duties  in  the  acquisition  cycle  were 
able  to  respond  to  all  questions  within  the  expected  range. 

The  validity  of  the  telephone  interview  schedule  was 
demonstrated  by: 

1.  In  preliminary  contact  with  knowledgeable  TO 
personnel,  the  names  most  often  mentioned  as  being  the 
"experts"  in  TO  Acquisition  were  Mr.  Art  Munguia,  of  the  Air 
Force  Institute  of  Technology,  School  of  Systems  and 
Logistics  and  Mr.  Chandler  H.  Weaver,  Chief  of  the  Technical 
Data  and  Management  Office,  Air  Force  Systems  Command, 
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Aeronautical  Systems  Division  (ASD)  and  CMSgt  Edward  J. 
j  Breslin,  Superintendent,  Technical  Order  Management  Agency, 

Armament  Division  (AD).  These  three  individuals,  because  of 
their  first  hand  knowledge  of  the  system,  were  consulted 
|  throughout  the  development  of  the  basic  research  questions 

and  again  were  consulted  when  evaluating  the  validity  of  the 
telephone  interview.  The  survey  instrument  used  in  this 
|  thesis  was  a  result  of  an  iterative  process  of  refinement  to 

elicit  responses  from  the  TO  manager  population  which  would 
permit  the  researchers  to  draw  valid  conclusions. 

2.  This  telephone  interview  was  further  validated  by 
the  Wigton  study.  Although  independently  developed,  the 
survey  instrument  used  by  Wigton  "to  resolve  day  to  day 
problems  encountered  by  Air  Force  TO  managers  during  the 
acquisition  of  TOs"  closely  paralleled  the  questions 
contained  in  this  thesis'  telephone  interview  schedule  (28). 

In  conclusion,  the  selection  of  a  telephone  interview 
as  a  measurement  tool  met  the  requirements  for  practicality, 
the  pretest  demonstrated  reliability,  and  validity  was  shown 
by  concurrence  of  the  experts. 

Research  Methodology 

The  telephone  interview  was  used  to  identify  the 
background  and  functions  of  those  individuals  performing  TO 
manager  duties.  The  interview  also  identified  possible 


solutions  to  problems  involving  TO  managers. 


Responses  to  the  interview  questions  were  groupea  by 
product  divisions  (AD,  ASD,  BMO,  ESD).  Questions  involving 
yes  or  no  responses  were  scored  by  percentage  and  validation 
of  the  questions  were  determined  by  establishing  if  a 
majority  of  the  respondents  agreed  or  disagreed.  Questions 
not  involving  yes  or  no  responses  were  also  evaluated  by 
what  percentage  of  the  respondents  had  common  responses. 
The  background  data  on  those  individuals  performing  TO 
manager  functions  within  the  product  division  was  used  to 
form  ?  composite  of  each  division.  The  composites  for  each 
product  division  were  compared  to  the  other  divisions.  The 
division  composite  were  then  assessed  to  determine  a 
composite  for  all  AFSC  TO  managers  on  a  question  by  question 
basis . 


Questions  1  through  27  were  used  to  form  the  TO 
manager  composite.  Questions  28  through  37  were  used  to 
gather  data  on  TO  management  improvement  areas  and  Question 
38  was  used  to  assure  completeness.  The  open-ended  nature 
of  the  question  was  intended  to  determine  if  any  possible 
problems  or  solutions  were  overlooked. 

Analysis  of  the  data  was  discussed  in  Chapter  V  of 
the  thesis.  When  the  telephone  survey  was  completed  and  the 
data  compiled  by  product  division  and  question,  evaluation 
was  accomplished  using  the  above  mentioned  technique  and 
then  reported  in  Chapter  IV,  Results. 


IV.  Results 


Chapter  Organization 

The  data  collected  during  the  telephone  interview  was 
examined  according  to  the  objectives  of  this  research.  The 
presentation  of  the  results  of  the  telephone  interview 
begins  with  a  description  of  the  survey  instrument. 
Following  the  survey  description,  a  tabular  representation 
of  the  data  received  in  response  to  the  37  telephone 
interview  questions  is  given  in  Tables  4.1  through  4.5. 
Composites  were  formed  for  each  of  the  product  divisions  and 
Air  Force  Systems  Command.  The  explanation  of  how  these 
composites  were  formed  is  provided  in  Appendix  B.  Finally, 
a  grouping  of  the  data  from  the  perspective  of  the  five 
research  questions  will  be  presented  in  Table  4.6  and  4.7. 

Survey  Description 

The  telephone  interview  was  organized  to  present  a 
composite  of  the  technical  order  manager  from  a  number  of 
perspectives.  Questions  1  through  4  were  designed  to  give 
biographical  information  on  the  specific  TO  manager  or 
program.  Questions  5  through  8  dealt  specifically  with 
those  qualifications  a  prospective  TO  manager  had  prior  to 
original  selection.  These  responses  concentrated  on 
management  training  and  education,  prior  job  training  and 
experience,  the  individuals'  perceptions  on  why  they  were 
selected  to  be  a  TO  manager  and  finally,  whether  they 


possessed  the  Special  Experience  Identifier  (SEI  300)  which 
indicates  a  prior  level  of  expertise  in  technical  order 
acquisition  and  management. 


Questions  9  through  12  examined  the  organizational 
structure  relative  to  the  individual  program.  Question  13 
asked  the  TO  managers  whether  they  perceived  top  management 
gave  sufficient  priority  to  the  acquisition  of  TOs.  The 
next  grouping  of  questions,  14  through  21  dealt  with 
specific  taskings  of  the  TO  manager  as  per  AF  policy  and 
directives.  TO  manager  interaction  with  other  individuals 
and  agencies  was  handled  by  questions  22  through  24. 
Questions  25,  26  and  27  provided  additional  information  on 
the  specific  TO  manager  and  program.  The  final  area  of 
concentration  was  improvement  ideas  and  this  was  dealt  with 
by  questions  28  through  37.  That  ended  the  structured 
portion  of  the  interview.  Completeness  was  assured  by 
question  38  which  was  an  open  ended  question  that  provided 
the  interviewee  a  chance  to  make  specific  recommendations 
and  suggestions  to  improve  the  management  of  technical 
orders . 

The  responses  for  Questions  1  through  37  are 
presented  in  tabular  form  shown  in  Tables  4.1  through  4.5. 
The  number  of  the  telephone  interview  question  is  located  in 
the  upper  left  corner  of  the  table.  The  total  number  of  A, 
B,  C,  D,  or  E  responses  and  their  coresponding  percentages 
are  listed  vertically  beneath  the  appropriate  product 
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division  heading  (i.e.  AD  -  Armament  Division).  The  far 
right  column  includes  total  responses  for  the  four  product 
divisions  under  the  heading  AFSC  -  Air  Force  Systems 
Command.  Blank  spaces  indicate  those  alternatives  that  were 
not  possible  responses  for  a  specific  question.  The 
expanded  responses  and  the  responses  to  Question  38  were 
listed  by  product  division  in  Appendix  C. 

TO  Manager  Composites 

Composites  of  TO  manager  responses  for  each  of  the 
four  AFSC  product  divisions  (AD,  ASD,  BMO,  ESD )  and  a 
composite  for  the  entire  TO  manager  population  of  AFSC  were 
compiled  to  show  the  majority  responses  to  survey  Questions 
1  though  27  (See  Appendix  B).  The  responses  with  the 
highest  percentage  were  selected  as  representative  of  the 
product  division  position  for  that  question  and  were 
included  in  the  composite.  For  situations  where  the 
percentages  were  identical,  all  identical  alternatives  to 
the  question  were  included  as  representative. 

For  questions  where  all  alternatives  had  a  percentage 
higher  that  80%,  all  alternatives  were  included  in  the 
composite.  The  data  given  by  the  composites  were  used  in 
the  evaluations  discussed  in  the  Analysis  and  the 
Conclusions  and  Recommendations  chapters. 
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TO  Management  Improvement  Areas 


The  responses  to  survey  questions  28  through  37 
depict  areas  of  TO  management  that  TO  managers  believe  may 
need  improvement.  Since  the  responses  to  these  questions 
represented  opinions  of  TO  managers,  they  were  seperated 
from  the  other  questions  and  grouped  together.  The 
responses  to  these  questions  are  shown  in  Table  4.5  by 
question. 

Research  Questions 

The  evaluation  of  the  five  research  questions 
required  that  the  responses  to  the  survey  be  grouped  by 
research  question  and  product  division/AFSC.  The  following 
are  the  five  research  questions: 

1.  How  are  TO  managers  initially  identified? 

2.  Can  the  establishment  of  selection  criteria  for 
for  TO  managers  improve  their  effectiveness? 

3.  Can  training  improve  the  effectiveness  of  the  TO 
manager? 

4.  Can  the  TO  managers'  interaction  with  the  other 
functional  offices  inside  and  outside  of  the 
System  Program  Office  be  changed  to  improve  the 
effectiveness  of  the  TO  manager? 

5.  How  do  TO  managers  perceive  the  level  of  top 
management  awareness  of  TO  acquisition  problems? 

Shown  in  Table  4.6  are  the  survey  questions  as  they 
apply  to  the  specific  research  questions.  Table  4.7  cross 
references  the  research  questions  with  the  applicable  survey 


questions . 


Summary 

The  results  of  the  telephone  interview  were  presented 
according  to  the  objectives  of  this  research.  No  evaluation 
or  explanation  of  the  data  was  accomplished,  other  than  that 
required  to  describe  how  the  data  were  tabulated.  The 
analysis  and  conclusions  of  this  research,  based  on  the  data 


gathered  in  the  telephone  interview,  will  be  presented  in 
the  following  chapters. 


V.  Analysis 


Chapter  Organization 

The  analysis  section  breaks  down  the  telephone 

interview  questions  under  the  broad  areas  of: 

Biographical  Information 
Qualifications  for  Selection 
Organizational  Structure 
Perception  of  Top  Management 
TO  Manager  Interaction 
TO  Management  Improvement  Areas 
TO  Manager  Tasking 

The  analysis  then  continues  with  a  question  by  question 
presentation  of  the  results. 

Biographical  Information  Analysis 

Telephone  Interview  Question  1 :  Number  of  months 

assigned  to  programs  in  TO  acquisition? 

Overall,  52.8%  of  the  respondents  were  assigned  to 

their  current  programs  for  more  than  24  months.  The 

majority  of  TO  managers  stay  with  the  program  for  a 

relatively  controlled  time  and  do  not  move  more  frequently 

than  two  years.  47.8%  have  less  than  two  years  on  their 

current  program.  This  indicates  the  ongoing  nature  of  the 

weapon  system  acquisition  process  with  new  programs 

beginning  and  personnel  from  completed  programs  being 

reassigned  to  new  programs  or,  as  in  the  case  of  enlisted 

military  members,  being  returned  to  their  primary  career 

field.  The  data  points  out  that  only  7.9%  have  less  than 

six  months  assigned  to  their  current  program.  This  small 

percentage  is  accounted  for  by  the  limited  number  of  new 
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programs  entering  the  acquisition  process  and  the  limited 

number  of  new  personnel  entering  the  acquisition  field. 

Telephone  Interview  Question  2 ;  Any  coordination  or 
training  with  predecessor  or  experienced  TO  manager? 

The  majority  of  respondents  said  no  to  this  question. 
Two  product  divisions,  ASD  and  ESD,  had  the  highest 
percentage  of  negative  responses,  70.7%  and  72.5% 
respectively.  The  majority  of  TO  managers  came  into  a 
program  whether  it  was  new  or  ongoing  without  the  benefit  of 
being  acclimated  to  the  program.  The  TO  acquisition 
responsibilities  passed  to  them  without  an  orderly  exchange 
of  information  on  what  had  been  done,  what  condition  the 
program  was  in,  or  what  needed  to  be  done  next. 

Additionally,  the  majority  of  TO  managers  were  not 
trained  on  the  peculiarities  of  the  product  the  TOs  were  to 
support,  what  agreements  and  compromises  had  previously  been 
negotiated  in  problem  areas,  or  how  the  TO's  requirements 
had  been  tailored  to  the  system  being  procurred. 

Armament  Division  was  the  only  product  division  with 
an  aggressive  management  policy  to  address  this  problem. 
Whenever  possible,  the  hiring  and  assignment  of  new 
personnel  was  designed  to  ensure  an  overlap  of  incoming  and 
outgoing  TO  managers,  by  program.  Where  this  was  not 
possible,  the  formal  training  program  for  new  personnel 
provided  for  "over  the  shoulder"  supervision  by  an 
experienced  TO  manager. 


Telephone  Interview  Question  3 :  Number  of  TO 

managers  assigned  to  your  position  since  program 
inception? 

This  question  was  designed  to  establish  a  measure  of 

the  continuity  within  a  particular  program.  Unfortunately, 

while  the  responses  indicated  in  most  cases  multiple  TO 

managers,  additional  ramifications  were  impossible  to 

determine  because  no  link  existed  between  causes  for  TO 

manager  turnover,  length  of  the  acquisition  program,  and  the 

question  of  civilian  continuity  versus  military  PCS  moves. 

BMO  was  seemingly  an  exception  to  the  multiple  TO  manager 

phenomena.  This  exception  was  explained  by  the  relative 

newness  of  current  BMO  acquisition  programs. 

Telephone  Interview  Question  4 :  Program  acquisition 
phase  when  the  first  TO  manager  was  assigned? 

The  acquisition  phase  which  the  majority  of 
respondents  cited  for  assignment  was  Full-Scale  Development 
at  36.6%.  Although  Full-Scale  Development  is  the  most 
numerically  large  percentage,  when  combined  with  the 
Conceptual  and  Demonstration  Validation  Phases,  the  total 
percentage  for  the  middle  three  phases  was  near  90%.  Only 
5%  were  identified  in  the  Pre-Conceptual  Phase  and  none  of 
this  5%  came  from  either  AD  or  BMO.  This  shows  a  trend  of 
little  upfront  planning  for  TOs  and  illustrates  the 
increased  attention  TOs  receive  during  the  later  phases 
prior  to  production. 


Qualifications  for  Selection  Analysis 


Telephone  Interview  Question  5 :  What  formal  management 
training/education  have  you  had? 

Less  than  14%  of  the  personnel  performing  TO 
management  had  a  bachelor's  degree  in  any  field  concerned 
with  management  and  the  percentage  of  masters  degrees  was  an 
even  smaller  8%.  While  the  possession  of  a  degree  in 
management  does  not  guarantee  an  ability  to  manage  technical 
order  acquisitions,  it  could  be  considered  as  at  least  an 
indicator  of  potential  aptitude. 

The  Air  Force  Institute  of  Technology  has  courses 
designed  for  Continuing  Professional  Education;  the 
interview  showed  that  over  50%  of  the  technical  order 
managers  had  taken  one  or  more  of  these  courses.  However, 
the  course  designed  specifically  for  TO  managers,  System  230 
was  cited  by  only  28  of  the  101  respondents.  The  other 
courses  which  combined  with  System  230  to  make  up  the  50% 
figure  were  primarily  logistical  in  scope.  ESD  had  the 
lowest  percentage  of  attendees.  Two  reasons  were  cited. 
Severe  undermanning  prevented  personnel  from  attending. 
Also,  ESD  personnel  were  responsible  for  all  logistics 
considerations.  Therefore,  when  the  time  could  be  found, 
the  course  attended  was  usually  related  to  logistics  rather 
than  TOs. 

The  ATC  courses  cited  were  primarily  NCO 
Academy/Leadership  School  and  represented  that  portion  of 
the  interview  population  which  either  is  or  was  enlisted. 


The  other  responses  fell  into  the  category  of 
management  course  work  at  less  than  a  bachelors  degree 
level . 

Telephone  Interview  Qu  e  £3 1  i.  on  6j_  What  prior  job 
experience  in  TO  related  fields  did  you  have  prior  to 
your  selection? 

The  majority  of  respondents  (67.3%)  said  their  job 
experience  with  TOs  was  as  a  maintenance  technician  at  the 
user  level.  Examination  of  the  product  divisions  found  this 
also  to  be  true.  A  high  percentage  (40%)  of  the  ESD  TO 
managers  said  that  they  also  had  experience  as  TO  managers 
in  the  past  prior  to  coming  to  their  present  program.  A 
small  number  of  respondents  said  they  had  experience  as 
AFPRO  representatives  (3.0%)  and/or  working  in  Test  and 
Evaluation  positions  (9.9%).  A  higher  number  (23.0%)  of  TO 
managers  said  they  haa  some  experience  as  non-acquisition  TO 
managers  either  in  AFLC  or  as  user  level  unit  TO  managers. 

The  range  of  prior  job  experience  in  TO  related 

fields  covered  a  great  span.  16  individuals  cited  no  prior 

job  experience  or  exposure  in  TO  related  fields  prior  to 

their  selection.  The  product  division  where  this  occurred 

with  the  greatest  frequency  was  ESD.  ESD  had  1  in  4 

individuals  with  no  prior  experience  in  TO  related  fields. 

Telephone  Interview  Question  7 ;  Why  do  you  feel  you 
were  selected  for  this  position? 

The  interview  question  results  showed  a  majority  of 
respondents  (64.3%)  said  they  were  selected  for  their 
current  position  by  management.  Selection  by  management 


meant  different  things  in  different  circumstances.  For 
example,  at  AD,  the  individual  in  most  cases  went  through  a 
formal  evaluation  of  his  technical  training,  APRs,  rank,  and 
several  other  factors  before  the  actual  hiring  (3).  At  ESD, 
selection  by  management  usually  meant  the  individual  was 
simply  available  and  unless  specific  problems  requiring 
special  or  extra  TO  expertise  had  developed,  one  TO  manager 
was  pretty  much  the  same  as  another. 

The  Special  Experience  Identifier  was  included  as  an 

alternative,  yet  no  one  gave  this  response  any  impact  on 

his  selection.  Twenty  one  TO  managers  said  they  were  "the 

next  person  through  the  door."  22.8%  volunteered  and  10.9% 

selected  career  broadening  as  their  choice.  In  response  to 

this  question,  some  respondents  combined  2  or  more  choices 

because  a  single  choice  did  not  create  a  complete  picture. 

Telephone  Interview  Question  » :  Do  you  have  the  Special 
Experience  Identifier  (SEI  300)? 

The  majority  of  respondents  (85.1%)  said  they  did  not 
have  the  SEI  300  ,  while  only  14.9%  said  they  did. 
Examination  of  the  product  divisions  results  also  showed  at 
least  one  TO  manager  in  every  five  possessed  the  SEI  except 
at  ESD.  Respondents  not  aware' of  the  SEI  300  identifier 
were  scored  as  no  responses. 


Organizational  Structure  Analysis 


Telephone  Interview  Question  9 :  Title  of  immediate 
supervisor? 

Sixty-three  percent  of  the  respondents  said  they  were 

supervised  by  the  DPML  of  the  program.  A  small  percentage 

(3%)  of  TO  managers  interviewed  were  supervised  by  either 

the  Program  Manager  or  the  Data  Management  Officer.  This 

would  indicate  that  the  management  of  TOs ,  a  logistics 

function,  was  actually  controlled  by  a  manager  of  logistics, 

the  DPML.  The  exception  was  the  Armament  Division.  Five  of 

the  more  senior  ranking  TO  managers  supervised  the  other  ten 

and  all  1 5  TO  managers  interviewed  at  AD  were  under  the 

control  of  the  manager  of  the  AD  central  TO  management 

agency.  Also  BMO  TO  managers  indicated  that  only  20%  of 

them  worked  for  the  DPML.  The  majority  of  these  TO  managers 

said  they  were  supervised  by  the  Human  Factors  Director. 

Telephone  Interview  Question  1 0 :  Number  of  personnel 
you  supervise? 

The  majority  of  respondents  (79.1%)  indicated  that 
they  supervised  no  one.  This  was  also  true  when  each 
product  division  was  examined.  Those  respondents  who  did 
say  they  supervised  other  personnel  were  TO  managers  who  had 
responsibility  for  the  TO  Management  Agency  (TOMA).  In  some 
programs  where  there  was  more  than  one  TO  manager  asigned, 
the  size  of  the  TOMA  ranged  from  two  to  eight  TO  managers 
and  one  TO  manager  provided  supervision  over  the  others. 


Telephone  Interview  Question  11 ;  Do  you  work  in  a  major 
or  non-major  acquisition  program? 

A  little  more  than  half  the  respondents  (52.5%)  said 

they  did  work  in  a  major  program  while  47.5%  said  they 

worked  in  a  non-major  program.  Two  product  divisions,  ASD 

and  BMO,  reported  a  majority  of  TO  managers  worked  in  major 

acquisition  programs  and  AD  TO  managers  said  93.3%  of  them 

worked  in  non-major  programs.  ESD  TO  managers  were  evenly 

split  between  major  and  non-major  programs. 

Telephone  Interview  Question  1 2 :  How  many  TOs  do  you 
manage? 

Of  the  TO  managers  interviewed,  44.6%  said  they 

managed  less  than  50  TOs,  33.7%said  they  managed  over  200 

TOs.  Within  the  ASD  product  division,  61%  of  the  TO 

managers  said  they  managed  over  200  TOs  with  one  TO  manager 

reporting  he  managed  1300  TOs.  The  majority  of  ESD 

respondents  (72.5%)  said  they  managed  less  than  50  TOs  and 

several  TO  managers  responded  they  managed  less  than  20  TOs. 

BMO  TO  managers  responded  that  60%  of  them  managed  between 

50  and  100  TOs.  Seventy-three  percent  of  the  AD  TO  managers 

said  they  managed  no  more  than  100  TOs.  A  concise  summary 

of  this  question  for  all  the  product  divisions  is  shown  in 

the  TO  Manager  Composites,  Appendix  B. 

Telephone  Interview  Question  2 5 :  What  portion  of  your 
duty  day/week  is  spent  performing  TO  management? 

The  majority  (49.5%)  of  TO  managers  responded  they 
spent  more  than  70%  of  their  time  performing  TO  management. 
This  was  also  true  for  AD,  ASD,  and  BMO  TO  managers.  ESD  TO 


managers  on  the  other  hand  said  the  majority  of  them  spent 
on  the  average  between  10%  and  30%  of  their  time  on  TO 


management.  They  did  say  though,  in  the  expanded  responses, 

that  during  major  milestones  in  TO  management  such  as  In- 

Process  Reviews,  or  when  TO  problems,  arose  they  spent  much 

of  their  duty  day/week  performing  TO  management.  ESD  TO 

managers,  unlike  the  majority  of  the  other  product  division 

TO  managers,  are  also  responsible  for  all  fifteen  elements 

of  logistics.  In  fact  those  individuals  performing  the  TO 

acquisition  and  management  functions  at  ESD  refer  to 

(  themselves  as  logisticians  not  TO  managers. 

Telephone  Interview  Question  26 :  Is  this  an  additional 
duty? 

TO  managers  by  a  large  majority  (94.1%)  said  that  TO 
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management  was  not  an  additional  duty.  Question  25  pointed 
out  that  ESD  TO  managers  only  spent  10-30%  of  their  time  on 
TO  management,  but  this,  as  Question  26  states,  was  not  an 
additional  duty  but  a  part  of  the  ESD  logisticians'  duties. 
The  six  other  respondents  who  said  TO  management  was  an 
additional  duty,  did  so  because  they  were  temporarily  taking 
over  the  TO  management  functions  until  a  TO  manager  could  be 
assigned. 

Telephone  Interview  Question  2 7 ;  Are  you  evaluated 
based  primarily  on  your  TO  management  duties? 

The  majority  of  respondents  (55.4%)  said  yes,  they 

were  evaluated  on  their  TO  management  duties.  Yet,  as 


pointed  out  in  Questions  25  and  26,  ESD  TO  managers  are  not 


evaluated  solely  on  their  TO  management  duties,  but  as 
logisticians.  Thus,  removing  ESD  responses  from  the  data, 
91.8%  of  the  TO  managers  for  AD,  ASD,  and  BMO  were  evaluated 
primarily  on  their  TO  management  functions. 

Perception  of  Top  Management 

Telephone  Interview  Question  1 3 :  Is  sufficient  priority 
afforded  by  top  management  for  the  acquisition  of  TOs? 

The  majority  of  respondents  (54.5%)  said  no  to  this 
question.  An  examination  of  the  product  division  results 
showed  that  a  majority  of  ESD  TO  managers  (57.5%)  perceived 
top  management  as  giving  sufficient  priority  to  TO 
acquisition.  100%  of  the  TO  managers  at  BMO  felt  there  was 
not  enough  emphasis  placed  on  TOs  by  top  management. 


TO  Manager  Interaction  Analysis 

Telephone  Interview  Question  22 :  SPO  functional  offices 
with  whom  you  have  frequent  contact? 

Interview  results  for  all  respondents  showed  that  TO 
managers  had  frequent  contact  with  the  PM,  DPML,  Data 
Management  Officer  and  contracting.  However,  by  examining 
the  product  division,  AD  TO  managers  had  frequent  contact 
only  with  the  PM  and  DPML.  ASD  TO  managers  had  frequent 
contact  with  all  SPO  functional  offices  while  BMO  TO 
managers  did  not  have  frequent  interaction  with  the  DPML. 
ESD  TO  managers  also  said  they  had  frequent  contact  with  the 
PM,  DPML  and  the  Data  Management  Officer  but  not  with 


contracting . 
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Telephone  Interview  Question  23 ;  Activities  outside  the 
SPO  with  whom  you  have  frequent  contact? 

This  question  was  asked  to  determine  the  interaction 
a  TO  manager  had  with  an  outside  agency  in  performing  TO 
management  tasks.  Interview  results  for  all  TO  managers 
showed  that  respondents  had  frequent  contact  or  interaction 
with  the  user,  ALCs,  ATC  and  the  contractor  but  not  with  the 
APPRO.  Examination  of  the  product  divisions'  results 
revealed  that  each  of  the  divisions  had  differences  as  to 
the  frequency  of  contact  with  individual  outside  agencies. 
These  differences  only  point  out  the  level  of  involvement  TO 
managers  had  with  an  outside  agency  in  performing  TO 
management. 

Telephone  Interview  Question  24 ;  Common  problems  in 
coordinating  requirements  with  outside  activities  and 
agencies? 

The  biggest  problem  all  TO  managers  said  they  had  in 
coordinating  TO  requirements  with  outside  activities  and 
agencies  was  scheduling  (50.5%).  Although  scheduling  had 
the  majority  of  responses,  the  other  alternatives  also  had  a 
relatively  large  number  of  responses  (see  Table  4.4 
Telephone  Interview  Questions  22-27).  Examination  of  the 
product  divisions  also  showed  this  selection  of  responses. 
AD,  ASD,  and  ESD  TO  managers  indicated  the  majority  of  their 
coordination  problems  were  scheduling  while  BMO  TO  managers 
said  coordination  with  the  SPOs  contracting  personnel  was 
their  major  problem.  The  results,  along  with  expanded 
responses,  tend  to  indicate  that  the  coordination  problems 


are  interrelated  and  that  singling  out  only  one  coordination 
problem  as  the  key  would  not  be  a  true  indication  of  TO 
manager  coordination  problems. 


TO  Management  Improvement  Areas  Analysis 

Telephone  Interview  Question  28 ;  Assignment  of  better 
qualified  personnel? 

The  majority  of  respondents  (76.2%)  said  that  there 
needs  to  be  the  assignment  of  better  qualified  personnel  in 
TO  management.  The  majority  of  TO  managers  in  the  product 
divisions  also  said  this  was  true,  but  AD  respondents  had 
the  lowest  percentage  of  responses  at  73.3%.  All  five  BMO 
TO  managers  said  the  TO  management  process  needs  better 
qualified  personnel. 

Telephone  Interview  Question  29 :  Formal  training  for  TO 
managers? 

The  majority  of  TO  managers  (94.1%)  said  there  needed 

to  be  formal  training  for  TO  managers.  Only  6  respondents 

said  formal  training  was  not  needed. 

Telephone  Interview  Question  30 ;  Higher  priority  to 
acquisition  of  technical  orders  by  top  management? 

The  highest  response  rate  to  this  question  was  yes 
with  65.3%.  All  four  product  divisions'  respondents  also 
said  there  should  be  a  higher  priority  for  the  acquisition 
of  TOs  by  top  management  but  45%  of  the  ESD  TO  managers  did 
not  think  that  this  was  an  area  that  needed  improvement. 
The  results  also  showed  that  all  five  BMO  TO  managers 
thought  this  was  an  area  that  needed  improvement. 
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Telephone  Interview  Question  3 1 ;  Improve  TO  manager 
guidance  as  outlined  in  AF  policy  and  procedure? 

This  question  refers  mainly  to  TO  00-5-1  and  the  Air 
Force  800  series  of  regulations.  Only  67.3%  of  the 
respondents  said  this  was  an  area  that  needed  improvement. 
32.7%  of  the  TO  managers  said  guidelines  and  regulations 
were  acceptable  as  they  were.  Examination  of  the  results 
for  the  individual  product  divisions  found  there  was  no 
difference  in  the  responses  than  was  found  for  the  entire  TO 
manager  population. 

Telephone  Interview  Question  32 :  A  TO  Handbook? 

The  majority  of  TO  managers  interviewed  (69.3%)  said 

there  needed  to  be  a  TO  manager's  handbook,  while  30.7%  said 

it  was  not  necessary.  These  results  also  reflected  similar 

percentages  within  each  of  the  product  divisions. 

Telephone  Interview  Question  33 :  Better  coordination 
between  AF  agencies  and  major  commands? 

The  results  to  this  question  showed  a  relatively 
slight  majority  (53.5%)  in  favor  of  improving  this  area  of 
TO  management.  On  the  other  hand,  46.5%  said  this  was  not 
so.  In  particular,  ESD  TO  managers  said  a  majority  of  them 
(55%)  believed  coordination  with  other  agencies  was 
sufficient  the  way  it  was.  Also,  the  difference  between 
responses  for  ASD  respondents  was  close.  51.2%  said  yes, 
while  48.8%  said  no. 

Telephone  Interview  Question  3 4 :  Career  field  for  TO 
managers? 

The  majority  of  the  TO  managers  responded  yes  (58.4%) 
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to  this  question  with  41.6%  responding  no  to  a  TO  manager 
career  field.  The  results  by  product  division  showed  that 
for  BMO  TO  managers,  100%  believed  there  should  be  a 
separate  career  field  while  a  majority  of  ESD  respondents 
(60%)  said  no,  there  should  not  be.  AD  results  showed  a 
slight  difference  favoring  the  career  field  with  ASD  TO 
managers  showing  a  73.2%  majority  in  favor  of  a  separate 
career  field. 

Telephone  Interview  Question  35 :  Central  Air  Force  TO 

Management  Agency? 

All  four  product  divisions  results  showed  a  majority 
of  respondents  favoring  the  idea,  with  an  overall  percentage 
rate  of  61.4%.  The  ASD  and  ESD  product  division  results 
showed  the  smallest  percentage  difference  between  the 
responses . 

Telephone  Interview  3  6  :  Special  Duty 

Identifier  (SDI)? 

77.2%  of  the  respondents  said  an  SDI  would  be  a  good 
TO  management  improvement  idea.  AD  respondents  had  the 
highest  percentage  of  yes  responses  with  93.3  while  ESD  TO 
managers  had  the  lowest  with  67.5%. 

Telephone  Interview  Question  37 :  Controlled  tour? 

The  majority  of  TO  managers  by  a  large  margin  (81.2% 
yes/18.8%  no)  said  a  controlled  tour  as  the  program  TO 
manager  was  necessary.  All  five  of  the  BMO  respondents  said 
a  controlled  tour  was  a  good  idea,  while  only  70%  of  the  ESD 
TO  managers  said  it  was  a  good  idea. 
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TO  Manager  Tasking  Analysis 

Telephone  Interview  Questions  14-21 : 

14.  Are  you  allowed  direct  input  to  SOW? 

15.  Do  you  identify  types  of  manuals? 

16.  Do  you  identify  types  of  documentation? 

17.  Do  you  approve  the  TO  Publication  Plan? 

18.  Do  you  approve  the  TO  Status  and  Schedules? 

19.  Do  you  identify  the  validation  requirements? 

20.  Do  you  identify  the  verification  requirements? 

21.  Do  you  arrange  for  the  printing  and  distribution  of 
TOs? 

These  eight  questions  were  grouped  together  because 
of  the  nature  of  the  results  below.  Therefore,  the  display 
format  was  required  to  be  different  for  this  section  of 
analysis. 

The  technical  order  manager  was  required  to  perform 
the  tasks  specified  in  TO  00-5-1  (14).  Telephone  interview 
questions  14  through  21  were  designed  to  give  the 
researchers  the  degree  of  task  compliance.  The  results  were 
not  as  expected.  The  researchers  expected  to  discover  non- 
compliance  in  these  taskings  and  the  level  of  non-compliance 
would  serve  as  an  indicator  of  less  than  optimum 
performance.  However,  with  one  exception,  the  responses  to 
questions  14  through  21  were  above  90%,  indicating  an 
extraordinarily  high  level  of  compliance.  The  one  exception 
was  question  21  dealing  with  the  printing  and  distribution 
of  technical  orders.  Question  21,  rather  than  indicating 
non-performance,  was  simply  a  reallocation  of  responsibility 
to  the  ALC. 
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Summary 

The  analysis  of  the  telephone  interview  questions  was 
presented  in  order  to  establish  a  basis  for  drawing 
conclusions  and  forming  recommendations.  These  conclusions 
and  recommendations  are  presented  in  the  final  chapter  of 


this  thesis. 


VI.  Conclusions  and  Recommendations 


Chapter  Organization 

This  chapter  states  the  conclusions  reached  by  this 
research  effort.  The  conclusions  are  based  on  information 
obtained  from  over  100  telephone  interviews  and  synthesized 
to  corespond  to  the  five  research  questions.  The  chapter 
continues  with  the  researchers'  recommendations  organized 
around  the  concept  of  a  Central  Technical  Order  Management 
Center  (CTOC),  and  cites  similarities  and  differences 
between  the  recommendations  of  this  research  effort  and  that 
of  its  predecessors.  Having  completed  these  tasks,  the 
chapter  concludes  with  recommendations  for  further  research. 

Research  Question  1_  Conclusions 

1.  How  are  TO  managers  initially  identified? 

There  presently  exists  no  single  standard  or 
universal  method  within  the  Air  Force  Systems  Command  to 
identify  individual  TO  managers.  Each  product  division  has 
unique  requirements  for  performance,  yet  none  of  the  product 
divisions  has  developed  a  method  to  identify  those 
individuals  who  could  meet  their  performance  requirements. 
Factors  such  as  having  .previously  been  a  user  level 
maintenance  technician,  while  applicable  in  all  product 
divisions,  were  not  a  common  factor  possessed  by  all  TO 
managers  within  any  product  division.  Experience  as  a  TO 
manager,  a  key  point  of  consideration  in  any  selection 


process,  is  not  even  an  identifiable  skill  under  the  present 
military  personnel  system.  For  example:  A  program  manager 
is  looking  for  an  individual  with  experience  as  a  radio 
repair  technician,  who  previously  worked  as  a  TO  manager  on 
another  program.  The  program  manager  wants  to  take  full 
advantage  of  the  trained  manpower  available.  The  present 
personnel  system  could  provide  him  with  a  list  of 
technicians  trained  in  radio  repair  but  nothing  would 
indicate  if  any  of  these  technicians  had  previously  been  TO 
managers . 

The  Special  Experience  Identifier  (SEI  300)  is 
perceived  by  some  people  as  answering  this  problem. 
However,  the  SEI  300  merely  represents  a  coding  on  the 
records  of  someone  curently  working  in  TO  management  and 
drops  from  the  system  when  that  individual  returns  to  his 
previous  career  field. 

The  civilian  identification  process  varies  from 
product  division  to  product  division.  Within  ESD  the 
individual  who  comes  to  work  on  a  program,  unless  he  is  a 
new  hire,  is  assumed  to  be  an  expert  in  each  of  the  15 
logistics  areas.  TO  management  is  just  one  area,  but  by 
virtue  of  the  fact  the  individual  has  worked  on  other 
programs,  he  is  now  also  an  experienced  TO  manager. 

The  one  common  element  in  each  product  division  is 


that  each 

has  one 

or  more  informally 

ident i f  ied 

"TO 

experts." 

These 

individuals,  by  a 

combination 

o  f 
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background,  training  and  experience,  have  become  the  point 
of  contact  for  other  TO  managers  within  that  product 
division.  The  usual  civilian  selection  process  is  more 
often  than  not  an  indication  of  availability  rather  than  an 
indication  of  ability.  The  ability  of  an  individual  is 
usually  passed  by  "word  of  mouth",  and  unless  special 
problems  arise  with  a  program  "the  next  person  through  the 
door"  can  become  the  person  responsible  for  TO  management. 
One  individual  commented  on  a  factor  which  makes  the 
identification  process  even  more  critical.  The  number  of 
experienced  TO  experts  is  declining.  The  group  of 
individuals  to  which  others  turn  for  advice  is  leaving  the 
system  due  to  retirement,  health,  and  other  considerations 
faster  than  they  can  be  replaced.  Common  sense  dictates 
that  a  one  for  one  replacement  of  experts  with  novices  will 
result  in  a  reduction  in  performance.  One  interviewee 
indicated  that  his  product  division  was  so  undermanned  that 
people  could  not  attend  training,  but  must  learn  on  the  job. 

Presently,  the  only  product  division  with  a  system  to 
identify,  monitor  and  track  experienced  TO  managers  is  AD. 
This  system  manages  "in  house"  a  list  of  names  of  qualified 
individuals  who  either  are  working  at  AD,  previously  worked 
at  AD,  or  who  have  been  recommended  to  work  at  AD. 

The  conclusion  that  there  is  no  established  method 
for  identifying  TO  managers  is  consistent  with  the  research 
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2.  Can  the  establishment  of  selection  criteria  for 
TO  managers  improve  their  effectiveness? 

The  first  chapter  of  this  thesis  stated  that  there 
were  no  formal  selection  criteria  for  TO  managers  and  the 
results  from  the  telephone  interview  supported  this 
statement.  Therefore,  the  answer  to  Research  Question  2 
drawn  from  this  research  is  "yes."  This  conclusion  is  based 
on  a  number  of  factors. 

First,  as  Research  Question  1  pointed  out,  there  must 
exist  a  means  to  identify  TO  managers.  Since  such  a  means 
does  not  exist,  it  is  difficult  to  build  a  pool  of  TO 
managers . 

Second,  there  is  currently  no  universally  applied 
selection  criteria  in  any  of  the  product  divisions.  Without 
a  defined  set  of  selection  criteria,  (i.e.  what  is  looked 
for  in  a  TO  manager)  choosing  the  right  person  for  the  job 
becomes  conjecture. 

Third,  this  research  identified  several  key  elements 
of  TO  management  which  can  be  used  to  establish  selection 


criteria  for  TO  managers.  These  key  elements  are: 

1.  Formal  management  training/education 

2.  Prior  TO  experience 

3.  Prior  management  experience 

4.  Size  and  complexity  of  program 

5.  Acquisition  phase  when  selection  of  a  TO  manager 
is  needed. 
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Fourth,  any  selection  criteria  must  correspond  with 
the  critical  taskings  of  the  TO  manager.  The  analysis  of 
Telephone  Interview  Questions  14  through  21  pointed  out  that 
all  TO  managers  performed  the  tasks  as  identified  in  TO  00- 
5-1.  Therefore,  the  potential  capability  of  an  individual 
to  perform  these  tasks  should  also  be  a  selection  criteria. 

Research  Question  _3  Conclusions 

3.  Can  training  improve  the  effectiveness  of  the  TO 
manager? 

This  research  effort  concludes  that  training  can 
improve  the  effectiveness  of  the  TO  manager.  To  begin  with, 
as  the  results  and  analysis  of  Telephone  Interview  Questions 
28  and  29  indicate,  the  majority  of  TO  managers  said  there 
is  a  need  for  better  qualified  personnel  and  formal  training 
programs  for  TO  managers.  These  results  are  supported  by 
the  findings  of  the  lack  or  inadequacy  of  training  at  the 
present. 

The  expanded  responses  (Appendix  C)  and  Telephone 
Interview  Question  38  pointed  out  it  takes  twelve  to 
eighteen  months  for  a  TO  manager  to  become  proficient. 
Since  nearly  50%  of  the  current  TO  managers  have  less  than 
two  years  experience,  they  have  just  reached  proficiency  in 
their  jobs.  Furthermore,  the  TO  managers  said  they  had 
little  or  no  coordination  or  training  with  either  their 
predecessor  or  an  experienced  TO  manager  when  they  came  to 
their  current  positions.  Also,  the  results  showed  only  28 


of  the  101  TO  managers  had  taken  the  AFIT  PCE  course;  SYS 


230,  Technical  Order  Acquisition  and  Management.  The 
majority  of  the  respondents  stated  their  only  previous  TO 
experience  was  as  user  level  maintenance  technicians.  These 
findings,  combined  with  the  fact  that  TO  managers  have  no 
extensive  training  in  management  principles,  support  the 
conclusion  that  training  can  improve  the  effectiveness  of 
the  TO  manager. 

Although  the  recommendation  section  of  this  research 
question  discusses  ways  to  improve  training  for  TO  managers, 
at  this  time  any  training  program  must  key  on  and  be 
developed  around  the  central  taskings  the  TO  managers  must 
perform  in  order  to  accomplish  effective  TO  management.  The 
results  and  analysis  of  Telephone  Interview  Question  14 
through  21  support  this  conclusion  in  that  for  all  product 
divisions,  TO  managers,  performed  the  TO  taskings  outlined 
in  TO-OO-5-1. 

Research  Question  4  Conclusions 

4.  Can  the  TO  manager's  interaction  with  the  other 
functional  offices  inside  and  outside  of  the  System 
Program  Office  be  changed  to  improve  the  effectiveness 
of  the  TO  manager? 

The  results  and  analysis  of  the  structured  Telephone 
Interview  Questions  indicated  that  the  TO  manager  had 
frequent  contact  with  the  other  functional  offices  of  the 
SPO  and  outside  the  SPO  with  agencies  and  commands.  These 
results,  combined  with  the  results  and  analysis  of  the  TO 


80 


manager  taskings,  lead  to  the  conclusion  that  from  a 
quantitative  analysis,  the  TO  manager's  interaction  with  the 
other  SPO  functional  offices  had  little  or  no  negative 
impact  on  the  effectiveness  of  the  TO  manager.  Thus,  these 
results  would  indicate  that  the  TO  managers'  interaction 
with  the  other  functional  offices  should  not  be  changed. 
Furthermore,  the  analysis  of  Telephone  Interview  Question  33 
showed  only  a  slight  majority  of  the  TO  managers  (53.5%  vs. 
46.5%)  were  in  favor  of  improving  the  coordination  between 
TO  manager  and  other  AF  agencies  and  major  commands.  Since 
coordination  is  a  key  part  of  the  interaction  process 
involving  the  TO  manager  and  the  other  functional  offices, 
this  result  only  gives  minor  support  to  the  conclusion  that 
the  interaction  process  should  not  be  changed.  However,  a 
review  of  the  expanded  responses  and  Telephone  Interview 
Question  38  responses  (Appendix  C)  indicate  that  the 
qualitative  interaction  of  the  TO  manager  and  the  other 
functional  offices  does  impact  upon  the  effectiveness  of  the 
TO  manager. 

The  qualitative  information  on  the  interaction 
process  indicates  there  is  a  negative  impact  upon  the  TO 
manager's  effectiveness.  Displayed  in  the  lists  below  is  an 
extraction  of  these  responses  from  Appendix  C. 

AD  Responses 

MAJCOMS  need  to  be  educated  on  time  constraints  on 

technical  data 


50%  of  TO  manager  inputs  to  SOW  are  changed  or  ignored 
without  consultation 


TO  managers  have  responsibility  for  TOs  but  no  authority 

user  needs  training  to  understand  acquisition  process 
for  TOs 

trouble  getting  through  AF  bureaucracy  to  person  at 
working  level 


ASP  Responses 

people  in  the  acquisition  business  outside  the  TO  system 
don't  know  what  TOs  are  all  about--  they  should 

AFLC  not  communicating  their  TO  needs  to  AFSC  people 

enlisted  TO  managers  don't  have  "horsepower"  to  get 
things  done 

DPMLs  and  PMs  should  take  AFIT  PCE  course  (SYS  230)  on 
TO  management  because  they  don't  understand  what  is 
involved  in  TO  acquisition 

military  TO  managers  should  be  the  rank  of  MSgt  at  least 

AFPRO  people  don't  understand  TO  acquisition  process 
enough  to  take  care  of  TO  validation 

TO  acquisition  definitely  needs  a  better  system  for 
coordination  and  communication 

TO  managers  need  to  use  tact  when  trying  to  get 
cooperation  from  others 

NCO  managers  find  it  difficult  to  function  because  of 
rank  differences  -  rank  seems  to  get  in  the  way 

education  of  others  on  the  TO  acquisition  process  is 
needed 

TO  managers  don't  have  the  authority  to  get  things  done 
the  way  they  should 


ESP  Responses 

people  in  acquisition  process  are  not  well  versed  in 
TOs;  no  one  understands  system 
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qualified  people  to  perform  verification  are  tough  to 
find 

process  is  dependent  on  ALC  actions;  as  a  result,  delays 
occur  which  the  TO  manager  has  no  control  over 

continuity  is  needed;  same  people  should  be  sent  to 
reviews  for  entire  process 

consistency  to  commitments  is  needed;  either  same  people 
or  at  least  commitment  to  predecessor's  position 

ALC  needs  to  put  more  emphasis  on  sending  right  people 
to  in-process  reviews,  verifications 

ALC  TDY  funds  are  not  earmarked  against  specific 
programs;  when  money  gets  tight  only  larger  more 
important  programs  get  money 

user  interface  is  needed  early;  need  user  to  send  same 
players  for  continuity 

DCAS/AFPRO  knowledge  of  TO  acquisition  process  very  poor 
to  the  point  of  contractor  validation  is  almost 
nonexistent 


BMP  Responses 

TO  manager  must  control  TO  portion  of  the  budget 

In  summary,  the  responses  to  the  structured  questions 
of  the  telephone  interview  indicate  that  the  TO  managers  had 
frequent  contact  with  the  other  functional  offices  in  the 
performance  of  their  TO  management  taskings.  Yet,  the 
quality  of  this  interaction,  as  indicated  in  the  expanded 
responses,  supports  a  conclusion  that  the  interaction 
should  be  changed. 


Research  Question  5  Conclusions 


The  results  and  analysis  of  Telephone  Interview 
Questions  13  and  30  showed  by  a  slight  majority  that 
TO  managers  perceived  the  level  of  top  management  awareness 
of  TO  acquisition  problems  to  be  inadequate. 

Although  Question  13  indicates  3  out  of  4  product 
divisions  thought  insufficient  priority  was  afforded  by  top 
management  for  acquisition  of  TOs,  the  only  product  division 
to  identify  this  with  an  overwhelming  majority  response  was 
BMO.  100%  of  BMO's  TO  managers  held  the  view  that 
insufficient  priority  was  afforded  to  acquisition  of  TOs  by 
top  management.  Due  to  the  small  size  of  TO  manager 
population  at  BMO,  this  high  percentage  did  little  to  sway 
the  figures  for  the  entire  population.  While  a  majority  of 
TO  managers  (54.5%)  felt  insufficient  priority  was  afforded 
to  acquisition  of  TOs  by  top  management,  the  researchers 
felt  that  this  was  too  small  a  margin  to  draw  any  definitive 
conclusions.  At  ESD,  however,  a  majority  felt  that 
sufficient  priority  was  afforded  to  the  acquisition  of  TOs 
by  top  management.  This  difference  in  perception  may  be 
accounted  for  by  the  fact  that  ESD  is  the  only  product 
division  where  TO  management  is  not  a  specialty. 

Question  30  responses  indicate,  in  contrast  to 
Question  13,  that  all  product  divisions  thought  a  higher 
priority  to  acquisition  of  TOs  by  top  management  might 
improve  TO  management.  The  conclusion  based  on  research 
results  is  that  the  level  of  top  management  awareness  is 


slightly  inadequate  and  is  a  lower  priority  area  for 
improvement.  The  recommendations  section  will  point  out 
proposed  reasons  for  the  inconclusive  nature  of  the 
responses  to  this  research  question. 

Recommendations 

As  pointed  out  by  Brown  and  Lyon,  any  proposed 
recommendations  to  the  TO  acquisition  process  must  apply  a 
systems  theory  approach  because  all  the  elements  or  factors 
are  integrated  and  interrelated  (7).  Therefore,  any 
recommendations  to  improve  the  effectiveness  of  the  TO 
manager  must  utilize  the  same  systems  approach  and  consider 
the  same  interrelationships.  The  factors  for  TO  manager 
effectiveness  are;  identification  of  TO  manager  personnel, 
the  criteria  used  for  their  selection,  the  means  used  for 
their  training,  the  method  of  management  organization  and 
the  level  of  top  management  support.  The  five  research 
questions  which  examined  these  factors  are: 

1.  How  are  TO  managers  initially  identified? 

2.  Can  the  establishment  of  selection  criteria  for 
TO  managers  improve  their  effectiveness? 

3.  Can  training  improve  the  effectiveness  of  TO 
managers? 

4.  Can  TO  managers'  i  nter  react  ion  with  other 
functional  offices  inside  and  outside  of  the 
System  Program  Office  be  changed  to  improve  the 
effectiveness  of  TO  managers? 

5.  How  do  TO  managers  perceive  the  level  of  top 
management  awareness  of  TO  acquisition  problems? 


The  cornerstone  of  any  recommendation  to  improve  the 
TO  acquisition  process  and  the  effectiveness  of  the 
individual  TO  manager  is  the  establishment  of  a  Central 
Technical  Order  Management  Center  (CTOC).  The  CTOC  would 
serve  in  the  capacity  of  a  staff  agency  for  the  central 
management  of  TOs  and  would  provide  guidance,  expertise  and 
assistance  to  TO  managers.  In  order  to  perform  in  this 
capacity,  the  CTOC  should  be  established  at  three  levels. 

The  first  and  highest  level  CTOC  would  be  at  the  Air 
Staff  level.  This  Air  Force  CTOC  would  provide  broad  TO 
management  policy  and  guidelines  for  the  entire  Air  Force, 
integrating  AFSC  and  AFLC  requirements  and  providing  an  Air 
Force  interface  to  DoD  needs  in  TO  management. 

The  second  level  CTOC,  located  at  HQ  AFSC,  would 
implement  the  Air  Staff  CTOC  policy  dealing  primarily  with 
the  acquisition  of  TOs  as  a  subset  of  the  weapon  system 
acquisition  process.  Also,  this  CTOC  would  provide  further 
AFSC  policy  and  guidance  relating  to  the  third  level  CTOC 
located  in  each  of  the  product  divisions. 

The  third  level  CTOC  would  be  located  in  each  of  the 
product  divisions.  The  third  level  CTOCs  would  implement 
higher  level  CTOC  policy  and  provide  guidance  and  expertise 
to  the  TO  managers  located  within  the  SPOs  of  the  product 
division.  Because  each  of  the  product  divisions  have 
different  needs  and  requirements  for  TOs,  the  product 


division  CTOC  s  management  policy  and  procedures  would  be 
tailored  to  meet  these  needs  and  requirements. 

The  concept  of  a  three  level  CTOC  should  provide 
increased  standardization  in  policy  and  guidance  throughout 
the  TO  system,  especially  within  AFSC,  and  still  be  able  to 
fulfill  the  individual  aquisition  program  needs  for  TO 
management. 

This  concept  is  in  full  agreement  with  the  results  of 
this  research  because  it  would  key  upon  the  central  figure 
in  TO  management,  the  TO  manager,  by  providing  guidelines, 
policy,  and  most  importantly,  expert  support.  The  CTOC 
concept,  as  discussed  here,  would  do  this  through  the 
management  of  the  five  TO  manager  effectiveness  factors; 
identification  of  TO  manager  personnel ,  their  selection, 
training,  organizational  and  top  management  support. 

Prior  research  studies  have  identified  possible 
solutions  to  TO  management  problems,  which  support  the 
improvement  of  the  five  TO  manager  effectiveness  factors. 
These  possible  solutions  are  discussed  below. 

The  concept  of  a  CTOC  or  some  central  control  agency 
for  the  management  of  TOs  and  the  TO  acquisition  process  is 
not  original  with  this  research.  Both  the  Brown  and  Lyon 
study  and  the  Wigton  study  recommend  the  creation  of  such  an 
agency  ( 7 ; 2  8 ) .  The  Wigton  study  concluded  that  a  CTOC 
should  be  a  great  asset  in  reducing  manning  problems,  and 
could  identify,  track  and  assist  program  offices  with 


Civ 


qualified  TO  managers  (28:23).  The  Brown  and  Lyon  study 
states  that  such  an  agency  would  provide  "expertise  and 
assistance  to  the  SPOs  and  technical  order  policy  makers 
from  a  centralized  location"  (7:85). 

A  functional  product  division  CTOC  is  already  in 
existence  at  AD  (4).  Mr.  Paul  Cambell  is  currently 
developing  a  product  division  CTOC  at  ESD  along  similar 
lines  (9).  The  potential  exists  for  a  product  division  CTOC 
at  BMO  due  to  an  organizational  structure  which  has  all  TO 
managers  co-located  in  the  same  office  and  reporting  to  the 
same  senior  manager.  ASD  has  the  highest  probability  of 
potential  gain  from  such  a  concept  due  to  their  large  size, 
number  of  TO  managers,  the  portion  of  the  acquisition  budget 
which  is  controlled  by  ASD,  and  the  fragmented  nature  of 
present  ASD  policy  and  guidance. 

This  research  differs  from  past  research  in  its 
recommendation  by  advocating  a  three  level  approach.  This 
three  level  approach  ensures  top  management  awareness  and 
subsequently,  support  of  TO  acquisition  problems. 

The  necessary  training  for  TO  managers  would  also  be 
based  on  the  three  level  CTOC  concept.  The  Air  Staff  level 
CTOC  would  provide  the  general  training  policy  guidelines, 
and  function  in  a  supervisory  capacity  to  ensure  adequate 
training  was  received  by  all  personnel  performing  TO 
acquisition  and  management. 


The  second  level  Air  Force  Systems  Command  CTOC  would 

be  responsible  for  the  development  and  administration  of  the 

broad  common  training  required  by  all  personnel  performing 

TO  management  and  acquisition.  This  research  recommends 

that  every  TO  manager  attend,  preferably  prior  to  assignment 

but  no  later  than  six  months  after  assignment,  management 

courses  teaching  the  following  subject  areas: 

Introduction  to  Acquisition  Management 
Technical  Order  Acquisition  and  Management 
Introduction  to  Acquisition  Logistics 

Further,  a  more  detailed  course  should  be  developed  as  a 

follow-on  to  Technical  Order  Acquisition  and  Management  to 

address  more  specific  facets  of  TO  management  (4). 

The  third  level  CTOC  located  at  the  respective 

product  divisions  must  continue  the  training  process  begun 

at  the  second  level  with  local  TO  courses  supplemented  by 

supervised  on-the-job  training. 

The  lack  of  an  effective,  integrated  training  program 

for  TO  managers  has  been  cited  in  virtually  all  research  on 

the  subject.  The  Peter's  study  commented,  "...personnel 

performing  duties  as  TO  managers  did  not  have  the  knowledge 

or  expertise  to  effectively  perform  their  duties 

...personnel  became  proficient  through  a  trial  and  error 

learning  process"  (1:A13-1).  Furthermore,  Work  Group  5  (TO 

Acquisition  Improvement)  Consolidated  Findings  recommended 

that  with  regards  to  training  that: 


1.  AF  SC/ALXD  and  AFALC  should  actively  manage  the 
training  quotas  for  AFIT  SYS  230,  and  ensure  that 


only  TO  managers  are  trained  until  such  time  as 
we  are  able  to  afford  to  send  other  people  to  the 
course. 

2.  AFIT  SYS  230  should  be  expanded  to  include 
instructions  on  how  to  use  DID's,  MIL  specs,  and 
CDRL  's . 

3.  AFIT  SYS  100  should  be  a  prerequisite  to  AFIT  SYS 

230.  [5:7] 

Additionally,  Bemrose  and  Carwise  found  in  1967,  there  was 

"an  insufficient  number  of  trained  and  qualified"  TO 

managers  based  on  their  research  (11:45).  In  1983,  Major 

Towner  in  his  thesis  on  acquisition  of  technical  orders 

linked  the  CTOC  concept  with  training  by  stating  the  lack  of 

such  a  center  "makes  it  difficult  for  inexperienced  TOMA 

personnel  to  learn  the  process"  (25:39).  The  Air  Force 

Logistics  Management  Center  findings  agreed  with  this 

research's  recommendation  that  all  "prospective  or  newly 

assigned  TOMA  personnel"  should  attend  a  TO  manager's 

course,  and  its  findings  were  instrumental  in  the  creation 

and  establishment  of  SYS  230  at  AFIT  (2:28). 

The  third  level  of  training  at  the  product  division 
is  already  in  existence  and  functioning  well  at  AD  (4).  A 
similar  training  approach  is  also  being  utilized  within  the 
larger  program  offices  at  ASD.  The  F-15  SPO  has  developed  a 
comprehensive  in  house  OJT  program  to  advance  newly  assigned 
personnel  to  the  required  skill  level  in  a  time  sequenced 
plan  individually  tailored  to  the  knowledge  and  training  of 
the  TO  manager  (27).  These  two  examples  show  that  the 
potential  exists  for  the  third  level  CTOC  training  concept, 


either  administered  directly  by  the  product  division  CTOC  as 
in  the  case  of  AD  or  supervised  by  the  product  division  CTOC 
and  administered  by  the  large  SPO  as  in  the  case  of  the  F- 


Two  of  the  five  independent  variables  which  emerged 
in  this  thesis  were  the  problems  of  identification  and  the 
criteria  used  to  select  individuals  for  TO  manager 
positions.  As  pointed  out  earlier  in  this  chapter,  a  means 
must  be  developed  to  identify  TO  managers  and  define  their 
selection  criteria.  The  Special  Experience  Identifier  (SEI 
300)  will  not  be  a  factor  in  rectifying  either  problem  for 
reasons  detailed  under  Research  Question  1  Conclusions.  The 
career  field  or  Air  Force  Specialty  Code  issue,  while 
gaining  wide  attention  in  the  literature  has  developed  two 
distinct  camps  among  the  population  interviewed  for  this 
research.  One  group  thinks  a  career  field  is  a  definite 
must  and  the  other  group  does  not  want  a  career  field  to  be 
formed.  The  researchers  wish  to  stress  the  point  that  if  a 
career  field  were  to  be  developed,  an  environment  which 
would  break  down  the  barriers  between  the  opposing  groups 
must  first  be  created.  The  CTOC  or  systems  approach  could 
be  applied  to  this  issue.  To  create  a  career  field  over 
the  objections  of  the  various  groups,  its  proponents  would 
have  to  be  located  at  least  in  the  Air  Staff  Level  CTOC. 
The  concerns  of  military  and  civilian  TO  managers  would  have 
to  be  dealt  with  equally  but  seperately.  The  career  field 


would  only  be  open  to  individuals  identified  as  specialists 
and  who  also  have  the  desire  to  be  in  the  field.  This  would 
permit  the  flexibility  required  at  the  product  division 
level  and  allow  the  diversity  necessary  in  a  dynamic 
acquisition  environment. 

The  designation  of  specialist  will  be  based  on  actual 
working  experience  as  a  TO  manager,  performance  as  a  TO 
manager,  individual  desires,  and  AF  needs.  The  product 
level  CTOC  will  make  the  recommendation  to  the  Air  Force 
Systems  Command  level  CTOC  that  an  individual  be  assigned 
the  Air  Force  Specialty  Code/career  field  or  its  civilian 
equivalent.  This  would  provide  career  progression  in  TO 
management,  something  currently  lacking  and  a  source  of 
several  negative  responses  to  the  question  on  creation  of  a 
career  field. 

This  thesis  strongly  recommends  a  Special  Duty 
Identifier  be  developed  as  a  prefix  or  suffix  to  every 
experienced  TO  manager's  present  Air  Force  Specialty  Code  or 
career  field.  In  this  manner,  the  experienced  individual 
can  be  tracked  and  identified  by  the  personnel  system.  The 
experienced  individual  is  defined  as  a  person  who  has 
performed  the  TO  acquisition  management  tasks  identified  by 
TO  00-5-1  for  a  minimum  of  three  years  and  has  completed  all 
required  training  as  specified  in  the  training 
recommendations . 


The  issue  of  selection  criteria  for  TO  managers  can 
now  be  tied  to  the  identification  process.  The  conclusion 
to  Research  Question  2  listed  five  key  elements  for  the 
basis  for  the  selection  criteria.  These  are: 

1.  Formal  management  training/education 

2.  Prior  TO  experience 

3.  Prior  management  experience 

4.  Size  and  complexity  of  program 

5.  Acquisition  phase  when  selection  of  a  TO  manager 
is  needed 

These  key  elements  and  policy  established  by  the 
proposed  AF  SC  CTOC  provide  the  product  division  CTOC 
guidelines  within  which  to  make  hiring  decisions  for  their 
TO  manager  positions.  As  guidelines,  they  allow  the  product 
division  CTOC  both  the  flexibility  and  judgement  in 
coordination  with  the  PM  or  DPML,  to  select  the  right  person 
for  the  right  job.  The  driving  force  for  the  selection  of 
an  Individual  for  a  TO  manager  position  is  to  match 
individual  qualifications  to  the  specific  needs  of  the 
program. 

Although  the  telephone  interview  responses  relating 
to  the  TO  manager's  interaction  with  other  offices  showed  a 
favorable  frequency,  the  expanded  responses  listed  in 
Research  Question  4  Conclusions  and  also  in  Appendix  C 
indicated  many  interaction  problems.  Therefore,  the 
recommendations  of  this  thesis  would  once  again  rely  on  the 
CTOC  concept  to  improve  the  quality  of  interaction  between 
offices  both  inside  and  outside  the  SPO.  Other  personnel 
within  the  weapons  system  acquisition  process  who  must 


interact  with  the  TO  manager  should  attend  training  to  fully 
understand  and  facilitate  communications  between  offices  of 
differing  disciplines.  The  interaction  problem  results 
because  a  TO  manager  has  the  responsibility  for  TO 
acquisition  but  in  many  cases  has  no  authority  to  implement 
required  actions. 

In  the  recommendations  section,  the  statement  was 
made  that  an  Air  Staff  level  CTOC  would  help  to  ensure  top 
management  awareness  of  TO  acquisition  problems.  Top 
management  awareness  of  TO  acquisition  problems  was 
historically  a  major  problem  in  the  management  of  TOs  as 
discussed  in  the  studies  by  Brown  and  Lyon,  Towner,  Carwise 
and  Bemrose  and  Wigton  (  7 ; 2 5 ; 1 1 ; 2 8  ) .  However,  recent  high 
level  attention  in  the  area  of  TOs  has  at  least  temporarily 
altered  the  previously  held  perception  of  top  management 
awareness  regarding  TOs  based  on  this  research's  findings. 

This  thesis  has  shown  that  TO  managers  perceive  they 
are  receiving  more  support  now  than  in  the  past.  But, 
further  support  is  necessary  to  obtain  future  improvements 
in  TO  management.  The  research  recommendation  for  a  three 
level  CTOC  would  provide  the  top  management  support  without 
which  the  needed  improvements  would  never  be  implemented. 
Top  management  support  is  the  key  to  any  improvement  in  the 
TO  acquisition  process. 


Further  Recommendations  for  Research 


The  first  recommendation  is  that  in  an  effort  to 
obtain  a  complete  picture  of  the  entire  TO  manager 
population,  a  study  be  made  of  the  Air  Logistics  Command 
personnel  performing  this  management  function. 

A  second  issue  requiring  research  is  the 
establishment  of  some  standard  of  measure  to  judge  and 
compare  TO  cost,  timeliness  and  useability  across 
acquisition  programs.  This  standard  of  measure  is  a 
necessity  in  establishing  accountability,  a  key  to  effecting 
any  positive  change. 

Third,  the  researcher's  felt  a  sense  of  shared 
frustration  with  previous  researchers  that  problems 
identified  as  early  as  1967  have  stil 1  had  no  corrective 
action  taken.  A  look  at  what  factors  contribute  to  this 
resistance  to  change  within  the  acquisition  community  could 
prove  to  be  revealing. 

Chapter  Summary 

This  study  offered  managers  and  policy  makers  in  the 
weapons  system  acquisition  process  information  about  the 
effectiveness  of  TO  managers  and  the  problems  which  impact 
that  effectiveness.  The  answers  toward  which  the  five 
research  questions  were  directed  present  viable  alternatives 
for  the  improvement  of  TO  manager  effectiveness.  The 


systems  approach  stresses  that  only  a  solution  which 
integrates  all  of  the  interrelated  facets  of  TO  management 


will  enhance  the  of  impact  the  TO  manager.  The  three  level 
Central  Technical  Order  Center  provides  such  a  solution. 
Additionally,  in  the  course  of  this  study,  other  questions 
have  been  raised.  Future  research  efforts,  answering  those 
questions,  will  aid  in  the  formation  of  a  more  effective  TO 
acquisition  process  and  thus  provide  better  TOs  for  the 


United  States  Air  Force. 


Appendix  A:  Telephone  Interview  Schedule 


INTRODUCTION:  Good  morning/afternoon.  I  am  Capt.  _ 

a  graduate  student  at  the  Air  Force  Institute  of  Technology. 
I  am  doing  research  concerning  the  TO  acquisition  process, 
and  you  can  help.  You  were  asked  to  participate  because 
your  duties  involve  TO  management.  Your  responses  to  my 
questions  will  be  kept  anonymous,  and  will  be  combined  with 
the  responses  of  others  who  have  similar  TO  management 
duties.  The  purpose  of  my  research  is  to  learn  more  about 
the  TO  acquisition  process  and  in  particular,  managements' 
impact  upon  the  process.  Your  responses  will  help  me  do 
this.  Please  consider  only  the  acquisition  process  for  new 
systems  when  giving  your  responses.  This  interview  consists 
of  a  series  of  structured  questions  and  one  open  ended 
question.  If  the  time  is  convenient  for  you  I  would  like  to 
begin.  (response  yes/no)  Do  you  have  any  questions  before 
we  start?  (Response  yes/no) 

Biographical  Information 

1 .  Number  of  months  assigned  to  program  in  TO  acquisition? 

a)less  than  6  b)6-12  c)12-18  d)18-24 

e)more  than  24 

2.  Any  coordination  or  training  with  predecessor  or 

experienced  TO  manager?  A)Yes _  B)No _ 

3.  Number  of  TO  managers  assigned  to  your  position  since 
program  inception? 

a)l  b)2  c)3  d)4  or  more  e) unknown 

4.  Program  Acquisition  Phase  when  first  TO  manager 
assigned? 

a )Pre-Conceptual  b) Conceptual  Exploration 

c ) Demonstration-Validation  d)Full-Scale  Development 
e ) Production 

Qualifications  for  Selection 

5.  What  formal  management  training/education  have  you  had? 
(you  may  select  more  than  one) 

a)  Bachelors  degree  in  Management 

b) Masters  degree  in  Management 

c )  AFIT  PCE  courses 

d) Air  Training  Command  courses 

e)  other  _ 


6.  What  prior  job  experience  in  TO  related  fields  did  you 
have  prior  to  your  selection?  (you  may  select  more  than 
one ) 

a ) Maintenance  (user)  Technician  b) previous  TO  manager 

c )non-acquisition  TO  manager  d)Test  and  Evaluation 
e) APPRO 

7.  Why  do  you  think  you  were  selected  for  this  position? 
a) "next  person  through  the  door"  b)volunteer 
c)SEI  d)career  broadening  e)selected  by  management 

8.  Do  you  have  the  Special  Duty  Identifier  (SEI  300)? 

A)  Yes _  B  ) No _ 

Program  Organizational  Structure 

9.  Title  of  immediate  supervisor? 

a) PM  b ) DPML  c ) DMO  d) other _ 

10.  Number  of  personnel  you  supervise? 

a)0  b ) 1  c)2  d)3  e)4  or  more 

11.  Do  you  work  in  a  major  or  non-major  acquisition  program? 
(Major  -  $200  million  RDT&E,  $1  billion  procurement) 

a) Major  b) Non-Major 

12.  How  many  TOs  do  you  manage? 

a ) less  than  50  b)50-10Q  c)100-150  d)150-200 

e)200  or  more 

To  the  following  questions  please  respond  Yes/No. 

Perception  of  Top  Management 

13.  Is  sufficient  priority  afforded  by  top  management  for 

acquisition  of  TOs?  A)Yes _  B)No _  Explain: 


TO  Manager  Taskings 

14.  Are  you  allowed  direct  input  to  SOW?  A) Yes _  B)No _ 

15.  Do  you  identify  types  of  manuals  (i.e.  mil-spec  vs 

commercial)  A ) Yes _  B)No _ 

16.  Do  you  identify  types  of  documentation?  (i.e.  mil-spec, 

mil-std.  DIDs,  AFADs )  A)Yes _  B)No _ 


17.  Do  you  approve  the  TO  Publication  Plan 
(TOPP)?  ( DI-M-3401 )  A ) Yes _ B)No _ 


18.  Do  you  approve  the  TO  status  and  schedules? 

(DI-M-3402)  A ) Yes _  B)No _ 

19.  Do  you  identify  the  validation  requirements? 

A)  Yes _  B)No _ 

20.  Do  you  identify  verification  requirements?  A) Yes _ 

B )  No _ 

21.  Do  you  arrange  for  the  printing  and  distribution? 

A)  Yes _  B)No _ 

TO  Manager  Interaction 

22.  SPO  functional  offices  with  whom  you  have  frequent 
contact?  (you  may  select  more  than  one) 

a )PM  b ) DPML  C ) DM0  d ) PCO 

23.  Activities  outside  the  SPO  with  whom  you  have  frequent 
contact?  (you  may  select  more  than  one) 

a) User  b ) ALC  c)ATC  d ) AFPRO  e) Contractor 

24.  Common  problems  in  coordinating  requirements  with 
outside  activities?  (you  may  select  more  than  one) 

a) lack  of  qualified  personnel  b)funds  c) schedule 

d ) other _ 

25.  What  portion  of  your  duty  day/week  is  spent  performing 
TO  management? 

a ) less  than  10%  b)10%-30%  c)30%-50%  d)50%-70% 

e (more  than  70% 

26.  Is  this  an  additional  duty?  A) Yes _  B)No _ 

27.  Are  you  evaluated  based  primarily  on  your 

TO  manager  duties?  A) Yes _  B)No _ 

What  of  the  following  areas  do  you  think  might  improve 
TO  management? 

Improvement  Areas 

28.  Assignment  of  better  qualified  personnel ?A) Yes _ 

B )  No _ 

29.  Formal  training  for  TO  managers?  A) Yes _  B)No _ 

30.  Higher  priority  to  acquisition  of  TOs  by  top 

management?  A) Yes _  B)No _ 

31.  Improve  TO  manager  guidance  as  outlined  in  AF  policy  and 

procedures?  A)Yes _  B)No _ 
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32.  A  TO  Handbook?  A) Yes 


B)No 


33.  Better  coordination  between  AF  agencies  and  major 

commands?  A)  Yes _  B)No _ 

34.  Career  field  for  TO  managers?  A)Yes _  B)No _ 

35.  Central  Air  Force  TO  management  agency?  A) Yes _  B)No _ 

36.  Special  Duty  Identifier  (SDI)?  A) Yes _  B)No _ 

37.  Controlled  tour?  A) Yes _  B)No _ 

This  ends  the  structured  portion  of  the  interview.  I  would 
also  like  to  ask  you: 

38.  Do  you  have  any  specific  suggestions  or  recommendations 
to  improve  the  management  of  tech  orders? 


Finally,  in  order  to  generate  as  complete  a  list  of 
personnel  performing  TO  manager  duties  as  possible,  could 
you  provide  me  with  the  names,  projects,  and  telephone 
numbers  of  any  TO  managers  with  whom  you  work  or  have 
contact? 


This  ends  the  interview.  Thank  you  very  much  for  your  time 
and  assistance. 


THE  ARMAMENT  DIVISION  TO  MANAGER: 


Biographical  Information 


1.  had  more  than  24  months  assigned  to  program 

2.  had  coordination  or  training  with  an  experienced  TO 
manager  or  predecessor 

3.  was  the  second  TO  manager  assigned  on  the  majority  of 
the  programs 

4.  was  first  assigned  either  during  the  Conceptual 
Exploration  or  Demonstration-Validation  Phase 


Qualifications  for  Selection 


5.  has  attended  NCO  leadership  school 

6.  was  previously  a  maintenance  technician  at  the  user 
level 

7.  was  selected  for  his  position  by  local  acquisition 
management 

8.  does  not  have  the  Special  Experience  Identifier  (SEI 
300) 

Organizational  Structure 

25.  spent  more  than  70%  of  his  duty  day  or  week  performing 
TO  management  or  acquisition 

26.  considered  TO  management  as  his  primary  duty 

27.  was  evaluated  primarily  on  TO  manager  duties 

9.  worked  under  the  supervision  of  another  TO  manager 

10.  supervised  no  personnel 

11.  worked  on  a  non-major  acquisition  program  (as  per  DoD 
Directive  5000.1) 


12.  managed  less  than  50  TOs 


ill 


Perception  of  Top  Management 

13.  perceived  TO  acquisition  was  not  afforded  sufficient 
priority  by  top  management 

TO  Manager  Taskings 

14.  made  direct  inputs  into  SOW  and/or  RFP 

15.  identified  types  of  manuals 

16.  identified  types  of  documentation 

17.  approved  the  TO  publication  plan 

18.  approved  the  TO  Status  and  Schedules 

19.  identified  validation  requirements  in  the  SOW 

20.  identified  verification  requirements 

21.  arranged  for  TO  printing  and  distribution 

TO  Manager  Interaction 

22.  had  frequent  contact  with  PM  and  DPML 

23.  had  frequent  contact  with  user,  ALC  and  the  contractor 

24.  had  a  common  problem  in  scheduling  activities  with 
outside  agencies 

THE  AERONAUTICAL  SYSTEM  DIVISION  TO  MANAGER: 

Biographical  Information 

1.  had  more  than  24  months  assigned  to  current  program 

2.  had  no  coordination  or  training  with  an  experienced 
TO  manager  or  predecessor 

3.  was  the  fourth  TO  manager  assigned  on  the  majority  of 
programs 


was  first  assigned  during  the  Full-Scale  Development 
Phase 


•ualif  ications  for  Selection 


2 

5.  attended  at  least  one  Professional  Continuing  Education 
Course  at  AFIT 

6.  was  previously  a  maintenance  technician  at  the  user 
level 

7.  was  selected  for  his  position  by  local  TO  acquisition 
management 

8.  does  not  have  the  Special  Experience  Identifier  (SEI 
300) 

Organizational  Structure 

25.  spent  more  than  70%  of  his  duty  day  or  week  performing 
TO  management  and  acquisition 

26.  considered  TO  management  as  his  primary  duty 

27.  was  evaluated  primarily  on  duties 

9.  worked  under  the  supervision  of  a  DPML 

10.  supervised  no  personnel 

11.  worked  on  a  major  acquisition  program  (as  per  DoD 
Directive  5000.1) 

12.  managed  more  than  200  TOs 

Perception  of  Top  Management 

13.  perceived  TO  acquisition  was  not  afforded  sufficient 
priority  by  top  management 

TO  Manager  Taskings 

14.  made  direct  inputs  into  SOW  and/or  RFP 

15.  identified  types  of  manuals 

16.  identified  types  of  documentation 

17.  approved  the  TO  Publication  Plan 

18.  approved  the  TO  Status  and  Schedules 


19.  identified  validation  requirements  in  the  SOW 

20.  identified  verification  requirements 

21.  arranged  for  TO  printing  and  distribution 


TO  Manager  Interaction 

22.  had  frequent  contact  with  PM,  DPML,  DMO  and  PCO 

23.  had  frequent  contact  with  user,  ALC,  ATC,  AFPRO  and  the 
contractor 

24.  had  a  common  problem  in  scheduling  activities  with 
outside  agencies 


THE  BALLISTIC  MISSILE  OFFICE  TO  MANAGER: 


Biographical  Information 

1.  had  more  than  24  months  assigned  to  current  program 

2.  had  no  coordination  or  training  with  an  experienced  TO 
manager  or  predecessor 

3.  was  the  first  TO  manager  assigned  on  the  majority  of 
programs 

4.  was  first  assigned  during  the  Conceptual  Exploration  or 
Demonstration-Validation  Phase 


Qualifications  for  Selection 

5.  attended  at  least  one  Professional  Continuing  Education 
Course  at  AFIT 

was  previously  a  maintenance  technician  at  the  user 
level 

was  selected  for  his  position  by  local  TO  acquisition 
management 


does  not  have  the  Special  Experience  Identifier  (SEI 


Organizational  Structure 


25.  spent  more  than  70%  of  his  duty  day  or  week  performing 
TO  management  and  acquisition 

26.  considered  TO  management  as  his  primary  duty 

27.  was  evaluated  primarily  on  TO  manager  duties 

9.  worked  under  the  supervision  of  a  Human  Factors 
Director 

10.  supervised  no  personnel 

11.  worked  on  a  major  acquisition  program  (as  per  DoD 
Directive  5000.1) 

12.  managed  between  50  and  100  TOs 
Perception  of  Top  Management 

13.  perceived  .TO  acquisition  was  not  afforded  sufficient 
priority  by  top  management 

TO  Manager  Taskings 

14.  made  direct  inputs  into  SOW  and/or  RFP 

15.  identified  types  of  manuals 

16.  identified  types  of  documentation 

17.  approved  the  TO  Publication  Plan 

18.  approved  the  TO  Status  and  Schedules 

19.  identified  validation  requirements  in  the  SOW 

20.  identified  verification  requirements 

21.  arranged  for  TO  printing  and  distribution 

TO  Manager  Interaction 

22.  had  frequent  contact  with  PM,  DM0  and  PCO 

23.  had  frequent  contact  with  user,  ALC  and  AFPRO 

24.  had  a  common  problem  in  dealing  with  PCO 


THE  ELECTRONIC  SYSTEM  DIVISION  MANAGER: 


Biographical  Information 

1.  had  more  than  24  months  assigned  to  current  program 

2.  had  no  coordination  or  training  with  an  experienced  TO 
manager  or  predecessor 

3.  was  the  first  TO  manager  assigned  on  the  majority  of 
programs 

4.  was  assigned  during  the  Full-Scale  Development  Phase 

Qualifications  for  Selection 

5.  attended  at  least  one  Professional  Continuing  Education 
Course  at  AFIT 

6.  was  previously  a  maintenance  technician  at  the  user 
level 

7.  was  selected  for  his  position  by  local  TO  acquisition 
management 

8.  does  not  have  the  Special  Experience  Identifier  (SEI 
300) 

Orqanizationa 1  Structure 

25.  spent  10  to  30%  of  his  duty  day  or  week  performing  TO 
management  and  acquisition 

26.  considered  TO  management  as  one  of  several  primary 
duties 

27.  was  not  evaluated  primarily  on  TO  manager  duties 

9.  worked  under  the  supervision  of  a  DPML 

10.  supervised  no  personnel 

11.  worked  on  either  a  major  or  non-major  acquisition 
program  (as  per  DoD  Directive  5000.1) 

12.  managed  less  than  50  TOs 


Perception  of  Top  Management 

13.  perceived  TO  acquisition  was  afforded  sufficient 
priority  by  top  management 

TO  Manager  Taskings 

14.  made  direct  inputs  into  SOW  and/or  RFP 

15.  identified  types  of  manuals 

16.  identified  types  of  documentation 

17.  approved  the  TO  Publication  Plan 

18.  approved  the  TO  Status  and  Schedules 

19.  identified  validation  requirements  in  the  SOW 

20.  identified  verification  requirements 

21.  arranged  for  TO  printing  and  distribution 

TO  Manager  Interaction 

22.  had  frequent  contact  with  PMr  DPML  and  DMO 

23.  had  frequent  contact  with  user,  ALC,  ATC,  and  the 
contractor 

24.  had  a  common  problem  in  scheduling  activities  with 
outside  agencies 

THE  AIR  FORCE  SYSTEMS  COMMAND  TO  MANAGER: 

Biographical  Information 

1.  had  more  than  24  months  assigned  to  current  program 

2.  had  no  coordination  or  training  with  an  experienced  TO 
manager  or  predecessor 

3.  was  the  first  TO  manager  assigned  on  the  majority  of 
programs 

4.  was  assigned  during  the  Full-Scale  Development  Phase 
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'ualif ications  for  Selection 


2 

5.  attended  at  least  one  Professional  Continuing  Education 
Course  at  AFIT 

6.  was  previously  a  maintenance  technician  at  the  user 
level 

7.  was  selected  for  his  position  by  local  TO  acquisition 
management 

8.  does  not  have  the  Special  Experience  Identifier  (SEI 
300) 

Organizational  Structure 

25.  spent  more  than  70%  of  his  duty  day  or  week  performing 
TO  management  and  acquisition 

26.  considered  TO  management  as  his  primary  duty 

27.  was  evaluated  primarily  on  TO  manager  duties 

9.  worked  under  the  supervision  of  a  DPML 

10.  supervised  no  personnel 

11.  worked  on  a  major  acquisition  program  (as  per  DoD 
Directive  5000 . 1  ) 

12.  managed  less  than  50  TOs 

Perception  of  Top  Management 

13.  perceived  TO  acquisition  was  not  afforded  sufficient 
priority  by  top  management 

TO  Manager  Taskings 

14.  made  direct  inputs  into  SOW  and/or  RFP 

15.  identified  types  of  manuals 

16.  identified  types  of  documentation 

17.  approved  the  TO  Publication  Plan 

18.  approved  the  TO  Status  and  Schedules 


19.  identified  validation  requirements  in  the  SOW 

20.  identified  verification  requirements 

21.  arranged  for  TO  printing  and  distribution 


TO  Manager  Interaction 

22.  had  frequent  contact  with  PM,  DPML,  DMO  and  PCO 

23.  had  frequent  contact  with  user,  ALC,  ATC,  and  the 
contractor 

24.  had  a  common  problem  in  scheduling  activities  with 
outside  agencies 


10 


9 


Interview  Question  38  Responses 


AD  Responses 

lack  of  a  central  AF  management  facility 

lack  of  identification  of  need  to  automate 

training  is  more  important  than  assignment  of  better 
qualified  personnel 

a  career  field  would  make  the  process  worse 
automate  at  once 

develop  local  management  policies  and  procedures  and 
ensure  dissemination  of  this  information 

trouble  getting  through  AF  bureaucracy  to  person  at 
working  level 

consolidated  TO  for  TO  managers;  single  source  of 
nonconflicting  information 

would  like  to  work  entire  program;  start  to  finish;  avoid 
second  guessing 

formal  training  doesn't  teach  what  you  need  to  know  to  do 
day  to  day  job 

career  field  would  provide  stability;  stay  with  program 
through  completion;  promote  and  test  based  on  day  to  day 
job 

central  agency  to  provide  standardization  and  remove 
contradictions  from  regulations 

-  a  TO  handbook  until  regulations  are  consolidated  and 
conflicting  regulations  removed 

career  field  should  be  for  experienced  NCOs  to  ensure 
user  experience 

controlled  tour  should  be  tied  into  acquisition  time 
rather  than  a  strict  3  or  4  year  tour 


biggest  problem  is  the  need  for  better  directives, 
guidance  and  regulations 


need  more  direct,  precise  guidelines;  present  guidelines 
too  ambiguous 

verification  needs  to  be  performed  by  3  to  5  level 
technicians  not  senior  NCOs 

need  AFSC  to  have  control  of  personnel;  TO  manager  should 
have  5  to  7  level  before  becoming  a  TO  manager 

TO  manager  needs  training  in  data  management, 
configuration  and  logistics 

do  away  with  paper  TOs 

top  management  needs  to  be  more  informed 

more  guidance  or  a  handbook  is  needed;  presently  no  one 
source  for  information;  conflicting  guidance 

need  a  formal  training  course  before  becoming  a  TO 
manager 

need  to  educate  MAJCOMMs  on  time  constraints  on  technical 
data 

creating  a  career  field  not  a  good  idea  because  you  would 
lose  user  experience 

need  a  Special  Duty  Identifier  and  a  controlled  tour  for 
continuity 

50%  of  TO  manager  inputs  to  SOW  are  changed  or  ignored 
without  consultation 

TO  manager  has  responsibility  for  TOs  but  no  authority 

only  senior  NCOs  with  user  background  should  be  TO 
managers 

need  formal  training  course  on  nuts  and  bolts  of  day  to 
day  job 

TOs  need  to  have  separate  contract  line  item  like 
hardware;  would  solve  priority  problem 

need  a  technical  school  and  career  field;  can  then  test 
for  promotion  on  TO  management  not  old  career  field 

need  career  field  with  a  controlled  tour  to  enable  same 
person  start  to  finish  to  establish  accountability 

attend  formal  training  early  not  after  already  doing  job 
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people  are  rotated,  moved  frequently  especially  people  on 
In-Process  Review  teams 

TOs  badly  integrated  into  maintenance  concept 

AFLC  not  communicating  their  TO  needs  to  AFSC  people 

too  much  of  the  TO  acquisition  is  subjective;  need  more 
objectivity  in  TO  acquisition  guideliines 

no  standards  for  TO  verification  or  validation 

enlisted  TO  managers  don't  have  "horsepower"  to  get 
things  done 

DPMLs  and  PMs  should  take  AFIT  PCE  course  (SYS  230)  on  TO 
management  because  they  don't  understand  what  is  involved 
in  TO  acquisition 

TO  manager  must  be  a  person  who  works  well  on  their  own 
self -motivated 
works  well  with  others 
is  an  effective  communicator 

need  more  civilians  as  TO  managers;  military  don't  stay 
with  the  program  long  enough  to  establish  "corporate 
memory" 

military  TO  managers  should  be  at  least  a  MSgt  in  rank 

civilian  TO  managers  provide  longevity  and  stability 

problem  with  TO  verification  is  you  can't  get  equipment 
when  needed 

enlisted  military  TO  manager  does  not  get  promoted  based 
on  TO  acquisition  knowledge  but  on  knowledge  of  his 
current  AFSC 

militay  TO  manager  must  have  user  experience  of  at  least 
9  to  10  years  before  becoming  a  TO  manager 

if  there  is  going  to  be  a  Central  Air  Force  TO  Management 
Agency,  it  should  be  headed  by  a  Colonel 

TOs  should  be  a  separate  line  item  on  contract 

TO  management  training  program  should  start  with 
introduction  of  TO  acquisition  management  prior  to 
starting  job;  then  following  a  period  of  9  to  12  months 
on  the  job,  a  more  indepth  course  on  details  of  TO 
acquisition  management 


TO  management  guidelines  and  regulations  written  in  too 
broad  terms;  need  specifics 

for  military  enlisted  TO  managers,  a  TO  management  AFSC 
should  be  for  the  top  three  NCO  grades 

AFPRO  people  don't  understand  TO  acquisition  process 
enough  to  take  care  of  TO  validation 

need  a  better  system  for  evaluating  the  prices  and  costs 
of  TOs 

definitely  need  better  system  for  TO  acquisition 
coordination  and  communication 

TO  manager  needs  to  use  tact  when  trying  to  get 
cooperation  from  others 

difficult  for  NCO  TO  manager  to  function  because  of  rank 
differences;  rank  seems  to  get  in  the  way 

need  more  guidelines  for  commercial  manuals 

need  standards  in  TO  system;  guidelines  and  regulations 
don't  provide  them 

need  more  training  courses  in  acquisition  business  not 
just  for  TOs 

need  more  education  of  others  on  the  TO  acquisition 
process 

TO  managers  don't  have  the  authority  to  get  things  done 
the  way  they  should 

do  too  much  busy  work,  therefore  don't  get  the  right 
things  done 

takes  18  months  on  the  job  to  be  good  at  TO  management 

because  someone  did  TO  management  for  airframes  doesn't 
mean  the  same  person  will  do  a  good  job  managing  TOs  for 
engines 

#1  problem  in  TO  management  is  accountability;  no  one  in 
the  chain  or  at  the  top  is  accountable  for  TOs;  no 
responsible  structure 

treating  TO  management  as  just  another  "logistic  element" 
is  another  downfall  of  the  system 


need  a  management  chain  that  knows  the  TO  acquisition 
business 


ESP  Responses 

contractor  constantly  slips  schedule  on  TOs 

need  to  identify  early  to  contractor  the  priority  of  TOs 

-  don't  want  AFSC  for  TO  managers 

people  should  attend  formal  training  after  a  2  or  3  month 
field  exposure 

TOs  are  low  priority  with  all  other  logistics  functions 

people  in  acquisition  process  are  not  well  versed  in  TOs; 
no  one  understands  system 

training  is  a  big  factor  across  the  board;  need  better 
training  or  more  knowledgeable  people 

a  career  field  would  be  over  kill;  TOs  are  an  integral 
part  of  the  complete  logistics  system 

TO  acqusition  is  easy  to  learn;  a  background  in  the 
technical  specialty  is  more  important  (ie.  radio  repair) 

need  more  emphasis  on  logistics  in  general;  TOs  in 
particular 

TOs  are  most  important  logistics  aspect  but  little  or  no 
emphasis  is  placed  on  their  acquisition 

TAC  letter  is  excellent;  emphasis  now  on 
engineering/schedule  until  user  screams  TO  is  inadequate 

technical  support  not  provided;  RFP  went  out  without 
review  of  technical  inputs 

training  is  necessary  but  when  short-handed  can't  spare 
time  to  go 

sole  source  contractors  tacked  shortages  in  other  areas 
to  the  cost  of  TOs 

need  life  cycle  and  cost  factors  for  TOs;  if  contractor 
uses  173-10,  pamphlet  guidelines  are  $630  per  page 


little  historical  data  available  on  TO  costs;  can't 
dispute  a  contractor 's  proposal 


need  definitive  guidance  on  data  requirements  and 
management  for  commercial  manuals  (ie.  computers) 

tough  to  find  qualified  people  to  perform  verification 

need  higher  priority  to  acquisition  by  top  management 
especially  if  staff  is  like  at  ESD 

can't  emphasize  enough  the  need  for  training 

an  AFSC  would  provide  for  career  development  and 
progression 

functional  area  training  after  2  months  OJT  to  avoid 
getting  inundated  with  terms 

unless  a  program  is  large  there  is  not  enough  work  for  a 
TO  specialist 

lose  logistics  continuity  if  you  specialize  only  on  TOs 

need  help  in  identifying  costs  of  TOs;  no  set  standards; 
varies  from  contractor  to  contractor 

short  military  tours  cause  problems;  huge  military 
turnover 

deployment  takes  priority;  TO  schedules  are  unrealistic 

process  is  dependent  on  ALC  actions  as  a  result  delays 
occur  over  which  TO  manager  has  no  control 

every  ILS  manager  is  expected  to  be  an  expert  in  all  15 
logistics  areas —  contractor  has  an  expert  in  each 

too  much  work  to  spare  people  for  training 

best  solution  is  to  have  a  TO  specialist;  can't  happen  at 
ESD  because  of  manning 

need  a  TO  handbook  with  specific  guidance 

need  a  higher  priority  on  manning  so  you  can  specialize 

need  to  establish  system  baselines 

TO  process  needs  to  start  early  and  not  be  put  off  to 
last  minute 

contractor  is  under  no  monetary  incentive  to  produce  good 
Os 


career  field  for  TO  mangers  too  narrow;  need  experience 
in  other  logistics  areas 

need  a  central  book  to  draw  everything  together 

need  more  teeth  in  SPO  to  hold  contractor  to  (DIDs) 
rather  than  deleting  them  to  save  money 

need  a  way  to  track  changes  from  in-process  reviews  to 
actual  change 

need  continuity;  same  people  should  be  sent  to  reviews 
for  entire  process 

need  one  central  agency  or  ALC  to  ensure  compliance  with 
MILSTD 

the  luxury  of  functional  specialty  causes  you  to  operate 
in  a  vacuum;  what  you  do  in  TOs  affects  other  logistics 
considerations;  if  you  do  them  all  you  have 
accountability 

relying  on  validation  results  by  contractor  is  an 
unnecessary  expense  because  verification  is  what  we  rely 
on 

existing  training  courses  are  a  regurgitation  of 
regulations;  judgement  is  required  to  tailor  regs  to  a 
specific  program 

everything  is  number  one  priority 

need  consistency  to  commitments;  either  same  people  or  at 
least  commitment  to  predecessor's  position 

high  turnover  in  program  office  with  military 

having  a  logistics  manager  totally  responsible  for  TO 
acquisition  is  a  mistake;  too  big,  no  training,  no 
background 

ALC  needs  to  put  more  emphasis  on  sending  right  people  to 
in-process  reviews,  verifications 

ALC  not  given  enough  money  to  fully  support  the  TOY 
requirements 

ALC  TDY  funds  not  earmarked  against  specific  programs; 
when  money  gets  tight  only  larger  more  important  programs 
get  money 

given  the  importance  of  TOs,  they  deserve  more  visibility 


"metal  on  the  ramp"  philosophy  causes  TOs  to  get  pushed 
to  the  side 

don't  like  to  go  to  reviews  and  see  different  faces  at 
each  review 

TO  acquisition  needs  to  be  done  uniformly  throughout  the 
product  division 

user  needs  to  send  same  people  to  all  in-process  reviews 

training  needs  to  be  upfront;  presently  no  experience, 
learn  by  doing  and  rely  on  experienced  loggies 

need  to  clarify  what  we  expect  from  contractors 

every  TO  manager  should  go  to  school 

need  clarification  on  whether  to  use  AFADs  or  not 

there  is  a  problem  when  dealing  with  small  contractors 
who  can't  or  don't  know  how  to  prepare  TOs 

logistics  and  TOs  need  more  people;  undermanned  at  least 
20% 

need  to  remove  the  perception  that  loggies  are  "second 
team" 

in  AFSC  TOs  there  is  not  right  kind  of  visibility 

TO  manager  can't  influence  design  of  equipment 

need  to  change  perception  that  loggies  are  not  educated 
or  retired  on  active  duty 

rewrite  regulations  to  make  duties,  responsibilities  and 
schedules  definite 

need  one  way,  one  guideline  for  all 

need  to  get  people  through  the  existing  training 
programs;  either  don't  get  sent  or  get  sent  late 

main  problem  is  not  enough  time  for  training 

need  to  move  towards  supportability 

need  a  single  source  of  information;  a  how  to,  what  to, 
when  to  handbook 

specifications  are  outdated;  don't  use  current  technology 


need  to  rewrite  specs  to  avoid  paying  for  same  thing 
twice 


need  a  formal  program  of  training  enroute  PCS  to  get 
whole  job  perspective 

ongoing  battle  on  whether  to  add  commercial  manuals  to 
our  system 

need  leverage  to  get  commercial  manuals  reviewed 

TDY  schedules  conflict;  need  to  be  in  two  different 
places  at  the  same  time 

TO  AF SC  too  specialized 

TO  acquisition  no  big  deal;  fairly  easy;  not  too  many 
problems 

early  reviews  are  a  waste  of  time;  basically  blank  pages 

difficult  to  get  away  for  training;  can't  afford  to  let 
him  go 

need  user  interface  early;  need  user  to  send  same  players 
for  continuity 

need  one  slant  or  set  of  rules  on  joint  service 
acquisitions 

a  new  person  cannot  reasonably  be  expected  to  support  TOs 
without  early  training 

need  a  comprehensive  checklist  to  know  what  you  are 
supposed  to  be  doing  at  a  given  point  in  time 

good  people  get  promoted  out  of  acquisition  or  go  to 
industry 

ESD  TO  acquisition  is  mostly  done  by  locals  because  of 
high  transient  rate 

the  experience  level  of  acquisition  people  has  gone  way 
down 

there  are  conflicting  priorities  and  TOs  usually  come  out 
last 


need  a  two  step  process  of  training  followed  by 
experience 


career  field  a  fantastic  idea;  problem  is  you  get  to  GS- 
11  and  die  in  TOs 


a  CTOM  would  do  the  most  good;  would  give  TOs  the 
visibility  they  need 

being  a  TO  expert  ruins  your  image;  must  divorce  yourself 
from  TOs  for  promotion 

TOs  only  play  a  minor  part;  no  attention  at  any  level 

commercial  manuals  should  not  be  a  part  of  the  TO  system 

need  people  dedicated  to  in-process  reviews  from  ALC  and 
using  commands 

DCAS/AFPRO  knowledge  of  TO  acquisition  process  very  poor 
to  the  point  that  contractor  validation  is  about 
nonexistent 

delay  TO  verification  until  actual  user  has  had  TOs  in 
hand  on  the  job  for  at  least  a  year  if  user  can  perform 
their  duties  with  unverified  TOs 

need  current,  updated  MILSPECs 

logesticians  have  too  many  things  to  do  to  be  good  at 
any  of  them;  there  should  be  TO  acquisition  management 
experts  verifying  the  TOs 

TO  manager  career  field  would  create  "tunnel  vision"  and 
this  hinders  good  TO  management 

TOs  should  not  be  finalized  until  they  have  been  out  in 
the  field  at  least  a  year;  leave  in  verified  format 


BMP  Responses 

must  get  logistics  upfront  in  the  acquisition  cycle 

TO  manager  must  control  TO  portion  of  the  budget 

TO  managers  should  have  specific  guidelines  and  a 
handbook  for  reference 

TO  managers  should  be  trained  and  have  a  Special  Duty 
Identifier 

contractors  are  not  getting  qualified  people  who  know  TOs 
lack  of  qualified  people  staying  in  TO  acquisition  jobs 


always  hearing  about  poor  TO  management;  need  more 
feedback  on  good  programs 

need  automation  in  TO  acquisition  management 

should  have  positions  so  that  TO  manager  career  field  can 
be  established;  currently  there  are  no  positions 

because  of  the  way  TOs  are  being  currently  acquired  we 
are  wasting  too  much  money  on  TOs 

too  many  varied  regulations;  they  need  to  be  boiled  down 
to  a  few 

need  an  expanded  training  course  on  TO  management;  more 
of  a  "how  to  do  it"  course 
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The  objective  of  this  research  was  to  examine  the  job 
performance  of  the  U.S.  Air  Force  technical  order  (TO) 
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accomplished.  Identification  of  personnel,  lack  of 
selection  criteria,  amount  of  formal  training,  interaction 
of  functional  offices,  and  the  level  of  top  management 
awareness  were  found  to  be  the  most  significant  issues 
facing  the  TO  manager.  TO  manager  composites  were  formed 
for  each  of  the  four  product  divisions  (Armament  Division, 
Aeronautical  Systems  Division,  Ballistics  Missile  Office, 
Electronics  Systems  Division).  Finally,  a  composite  for  the 
entire  Air  Force  Systems  Command  was  formed. 

Recommendations  to  improve  the  effectiveness  of  the 
TO  manager,  barbed  on  the  formation  of  a  three  leveled 
Central  Technical  Order  Management  Center  (CTOC),  were 
presented,  within  this  three  leveled  CTOC  framework,  the 
following  recommendations  were  made: 

1.  Thedas  tab  lishment  of  *  career  fieldyifor  senior 
TO  acqusition  personnel  based  on  experience  and 
performance  within  the  field. 

2.  The  =identif  ication  of  TO  managers  by  utilizing  a 
Special  Duty  Identifier.  (SDI). 

3.  ThdHestablishment  of  selection  criteria,.based  on 
formal  management  training  and  education;  prior 
TO  and  management  experience;  and  the  size, 
complexity,  and  acquisition  phase  of  the 
program. 

4.  The  utilization  of  an  integrated  system  of  TO 
..  acquisition  management  courses  and  on-the-job 

training.,  t 

5.  The  contintradon  of  recent  attention  and 
-addurtionair  support  by  top  management,  to  the 
problems  of  TO  acquisition. 
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